• Questing for Quality Estimation A User Study

      Escartin, Carla Parra; Béchara, Hanna; Orăsan, Constantin (de Gruyter, 2017-06-06)
      Post-Editing of Machine Translation (MT) has become a reality in professional translation workflows. In order to optimize the management of projects that use post-editing and avoid underpayments and mistrust from professional translators, effective tools to assess the quality of Machine Translation (MT) systems need to be put in place. One field of study that could address this problem is Machine Translation Quality Estimation (MTQE), which aims to determine the quality of MT without an existing reference. Accurate and reliable MTQE can help project managers and translators alike, as it would allow estimating more precisely the cost of post-editing projects in terms of time and adequate fares by discarding those segments that are not worth post-editing (PE) and have to be translated from scratch. In this paper, we report on the results of an impact study which engages professional translators in PE tasks using MTQE. We measured translators? productivity in different scenarios: translating from scratch, post-editing without using MTQE, and post-editing using MTQE. Our results show that QE information, when accurate, improves post-editing efficiency.
    • Reader and author gender and genre in Goodreads

      Thelwall, Mike (Sage, 2017-05-01)
      There are known gender differences in book preferences in terms of both genre and author gender but their extent and causes are not well understood. It is unclear whether reader preferences for author genders occur within any or all genres and whether readers evaluate books differently based on author genders within specific genres. This article exploits a major source of informal book reviews, the Goodreads.com website, to assess the influence of reader and author genders on book evaluations within genres. It uses a quantitative analysis of 201,560 books and their reviews, focusing on the top 50 user-specified genres. The results show strong gender differences in the ratings given by reviewers to books within genres, such as female reviewers rating contemporary romance more highly, with males preferring short stories. For most common book genres, reviewers give higher ratings to books authored by their own gender, confirming that gender bias is not confined to the literary elite. The main exception is the comic book, for which male reviewers prefer female authors, despite their scarcity. A word frequency analysis suggested that authors wrote, and reviewers valued, gendered aspects of books within a genre. For example, relationships and romance were disproportionately mentioned by women in mystery and fantasy novels. These results show that, perhaps for the first time, it is possible to get large scale evidence about the reception of books by typical readers, if they post reviews online.
    • The reading background of Goodreads book club members: A female fiction canon?

      Thelwall, Mike; Bourrier, Karen (Emerald, 2019-09-09)
      Purpose - Despite the social, educational and therapeutic benefits of book clubs, little is known about which books participants are likely to have read. In response, this article investigates the public bookshelves of those that have joined a group within the Goodreads social network site. Design/methodology/approach – Books listed as read by members of fifty large English language Goodreads groups - with a genre focus or other theme - were compiled by author and title. Findings – Recent and youth-oriented fiction dominate the fifty books most read by book club members, while almost half are works of literature frequently taught at the secondary and postsecondary level (literary classics). Whilst JK Rowling is almost ubiquitous (at least 63% as frequently listed as other authors in any group, including groups for other genres), most authors, including Shakespeare (15%), Goulding (6%) and Hemmingway (9%), are little read by some groups. Nor are individual recent literary prize-winners or works in languages other than English frequently read. Research limitations/implications – Although these results are derived from a single popular website, knowing more about what book club members are likely to have read should help participants, organisers and moderators. For example, recent literary prize winners might be a good choice, given that few members may have read them. Originality/value – This is the first large scale study of book group members’ reading patterns. Whilst typical reading is likely to vary by group theme and average age, there seems to be a mainly female canon of about 14 authors and 19 books that Goodreads book club members are likely to have read.
    • Recepción en España de la literatura africana en lengua inglesa: generación de datos estadísticos con la base de datos bibliográfica especializada BDÁFRICA

      Fernández Ruiz, MR; Corpas Pastor, G; Seghiri, M (Fundacio per la Universitat Oberta de Catalunya, 2018-11-20)
      El presente artículo examina la recepción de la literatura africana en lengua inglesa en España basándonos en BDÁFRICA, una base de datos bibliográfica que recoge obras de autores nacidos en África y publicadas en español y en España entre 1972 y 2014. Se ofrece una reflexión crítica de las dificultades para definir la literatura africana como objeto de estudio, debido a su complejidad y heterogeneidad. Se propone, además, un conciso recorrido historiográfico por la conformación del canon de dicha literatura que se ha realizado desde Occidente. Asimismo, se demuestra la falta de estudios estadísticos sobre la recepción de literatura africana en lengua inglesa en España. Respondiendo a esta necesidad, el objetivo del artículo es detallar y analizar los datos estadísticos inéditos que proporciona la base de datos, adoptando una metodología descriptiva. Los resultados de este estudio, que aporta datos cuantitativos y cualitativos fiables y novedosos, son originales en tanto en cuanto reflejan y señalan los problemas de la traducción de la literatura africana en lengua inglesa en España. BDÁFRICA, que es gratuita y está disponible en red, pretende ser un recurso y una fuente que estimule el desarrollo de la investigación en literatura poscolonial en España. Sin duda, esta base de datos bibliográfica especializada es una herramienta muy valiosa, especialmente para investigadores, traductores y editoriales interesados en literatura africana.
    • Recursos documentales para la traducción de seguros turísticos en el par de lenguas inglés-español

      Corpas Pastor, Gloria; Seghiri Domínguez, Miriam; Postigo Pinazo, Encarnación (Universidad de Málaga, 2007-04-05)
      Las páginas que siguen a continuación resumen parte de la investigación realizada en el marco de un proyecto de I+D interdisciplinar e interuniversitario sobre Tecnologías de la Traducción, denominado TURICOR (BFF2003-04616, MCYT), cuyos objetivos principales son la compilación virtual de un corpus multilingüe de contratación turística a partir de recursos electrónicos y el desarrollo de un sistema de generación de lenguaje natural (GLN), también multilingü. El corpus Turicor alberga, pues, diversos tipos de documentos relativos a la contratación turística en las cuatro lenguas implicadas (español, inglés, alemán e italiano). En concreto, la tipologíatextual que ha vertebrado la selección de los documentos que integran los distintossubcorpus de los que consta Turicor abarca lo siguiente: legislación turística (internacional, comunitaria y nacional de los respectivos países incluidos); condiciones generales, formularios y contratos turísticos.
    • Refined Salience Weighting and Error Analysis in Anaphora Resolution.

      Evans, Richard (The Research Group in Computational Linguistics, 2002)
      In this paper, the behaviour of an existing pronominal anaphora resolution system is modified so that different types of pronoun are treated in different ways. Weights are derived using a genetic algorithm for the outcomes of tests applied by this branching algorithm. Detailed evaluation and error analysis is undertaken. Proposals for future research are put forward.
    • Register-Specific Collocational Constructions in English and Spanish: A Usage-Based Approach

      Pastor, Gloria Corpas (Science Publications, 2015-03-01)
      Constructions are usage-based, conventionalised pairings of form and function within a cline of complexity and schematisation. Most research within Construction Grammar has focused on the monolingual description of schematic constructions: Mainly in English, but to a lesser extent in other languages as well. By contrast, very little constructional analyses have been carried out across languages. In this study we will focus on a type of partially substantive construction from the point of view of contrastive analysis and translation which, to the best of our knowledge, is one of the first studies of this kind. The first half of the article lays down the theoretical foundations of the study and introduces Construction Grammar as well as other formalisms used in literature in order to provide a construal account of collocations, a pervasive phenomenon in language. The experimental part describes the case study of V NP collocations with disease/enfermedad in comparable corpora in English and Spanish, both in the general domain and in the specialised medical domain. It is provided a comparative analysis of these constructions across domains and languages in terms of token-type ratio (constructional restriction-rate), lexical function, type of determiner, frequency ranking of the verbal collocate and domain specificity of collocates, among others. New measures to assess construal bondness will be put forward (lexical filledness rate and individual productivity rate) and special attention will be paid to register-dependent equivalent semantic-functional counterparts in English and Spanish and mismatches.
    • A report on the Third VarDial evaluation campaign

      Zampieri, Marcos; Malmasi, Shervin; Scherrer, Yves; Samardžić, Tanja; Tyers, Francis; Silfverberg, Miikka; Klyueva, Natalia; Pan, Tung-Le; Huang, Chu-Ren; Ionescu, Radu Tudor; et al. (Association for Computational Linguistics, 2019-12-31)
      In this paper, we present the findings of the Third VarDial Evaluation Campaign organized as part of the sixth edition of the workshop on Natural Language Processing (NLP) for Similar Languages, Varieties and Dialects (VarDial), co-located with NAACL 2019. This year, the campaign included five shared tasks, including one task re-run – German Dialect Identification (GDI) – and four new tasks – Cross-lingual Morphological Analysis (CMA), Discriminating between Mainland and Taiwan variation of Mandarin Chinese (DMT), Moldavian vs. Romanian Cross-dialect Topic identification (MRC), and Cuneiform Language Identification (CLI). A total of 22 teams submitted runs across the five shared tasks. After the end of the competition, we received 14 system description papers, which are published in the VarDial workshop proceedings and referred to in this report.
    • Research dissemination and invocation on the Web

      Thelwall, Mike (MCB UP Ltd, 2002)
      The importance of the Web as a new medium for disseminating and promoting scholarly research is discussed. Particular attention is paid to its potential to provide evidence of wider impact for research than that which can be shown by citation analysis. Recommendations are made for basic strategies for the reporting of the online impact of research leading to the production of what is termed a Web Invocation Portfolio. A conceptual framework is also proposed to help funding and promotion committees assess and compare portfolios.
    • Research note: in praise of Google: finding law journal Web sites

      Thelwall, Mike (MCB UP Ltd, 2002)
      Google is a highly regarded and widely used search engine, particularly in academia. In this short note we comment on its remarkable ability to find journal Web sites, using a case study of law.
    • ResearchGate Articles: Age, Discipline, Audience Size and Impact

      Thelwall, Mike; Kousha, Kayvan (Wiley-Blackwell, 2016-03-28)
      The large multidisciplinary academic social web site ResearchGate aims to help academics to connect with each other and to publicise their work. Despite its popularity, little is known about the age and discipline of the articles uploaded and viewed in the site and whether publication statistics from the site could be useful impact indicators. In response, this article assesses samples of ResearchGate articles uploaded at specific dates, comparing their views in the site to their Mendeley readers and Scopus-indexed citations. This analysis shows that ResearchGate is dominated by recent articles, which attract about three times as many views as older articles. ResearchGate has uneven coverage of scholarship, with the arts and humanities, health professions, and decision sciences poorly represented and some fields receiving twice as many views per article as others. View counts for uploaded articles have low to moderate positive correlations with both Scopus citations and Mendeley readers, which is consistent with them tending to reflect a wider audience than Scopus-publishing scholars. Hence, for articles uploaded to the site, view counts may give a genuinely new audience indicator.
    • ResearchGate versus Google Scholar: Which finds more early citations?

      Thelwall, Mike; Kousha, Kayvan (Springer, 2017-04-26)
      ResearchGate has launched its own citation index by extracting citations from documents uploaded to the site and reporting citation counts on article profile pages. Since authors may upload preprints to ResearchGate, it may use these to provide early impact evidence for new papers. This article assesses the whether the number of citations found for recent articles is comparable to other citation indexes using 2675 recently-published library and information science articles. The results show that in March 2017, ResearchGate found less citations than did Google Scholar but more than both Web of Science and Scopus. This held true for the dataset overall and for the six largest journals in it. ResearchGate correlated most strongly with Google Scholar citations, suggesting that ResearchGate is not predominantly tapping a fundamentally different source of data than Google Scholar. Nevertheless, preprint sharing in ResearchGate is substantial enough for authors to take seriously.
    • Results from a web impact factor crawler

      Thelwall, Mike (MCB UP Ltd, 2001)
      Web impact factors, the proposed web equivalent of impact factors for journals, can be calculated by using search engines. It has been found that the results are problematic because of the variable coverage of search engines as well as their ability to give significantly different results over short periods of time. The fundamental problem is that although some search engines provide a functionality that is capable of being used for impact calculations, this is not their primary task and therefore they do not give guarantees as to performance in this respect. In this paper, a bespoke web crawler designed specifically for the calculation of reliable WIFs is presented. This crawler was used to calculate WIFs for a number of UK universities, and the results of these calculations are discussed. The principal findings were that with certain restrictions, WIFs can be calculated reliably, but do not correlate with accepted research rankings owing to the variety of material hosted on university servers. Changes to the calculations to improve the fit of the results to research rankings are proposed, but there are still inherent problems undermining the reliability of the calculation. These problems still apply if the WIF scores are taken on their own as indicators of the general impact of any area of the Internet, but with care would not apply to online journals.
    • Reuse of scientific data in academic publications

      He, Lin; Nahar, Vinita (Emerald Group Publishing Limited, 2016-07-18)
      Purpose In recent years, a large number of data repositories have been built and used. However, the extent to which scientific data is reused in academic publications is still unknown. This article explores the functions of re-used scientific data in scholarly publication in different fields. Design/methodology/approach To address these questions, we identified 827 publications citing resources in the Dryad Digital Repository (DDR) indexed by Scopus from 2010 to 2015. Findings The results show that: (i) the number of citations to scientific data increases sharply over the years, but mainly from data-intensive disciplines, such as Agricultural, Biology Science, Environment Science and Medicine; (ii) the majority of citations are from the originating articles; (iii) researchers tend to reuse data produced by their own research groups. Research limitations/implications data may be re-used without being formally cited. Originality/value The conservatism in data sharing suggests that more should be done to encourage researchers to re-use other’s data.
    • RGCL at GermEval 2019: offensive language detection with deep learning

      Plum, A; Ranasinghe, Tharindu; Orasan, Constantin; Mitkov, R (German Society for Computational Linguistics & Language Technology, 2019-10-08)
      This paper describes the system submitted by the RGCL team to GermEval 2019 Shared Task 2: Identification of Offensive Language. We experimented with five different neural network architectures in order to classify Tweets in terms of offensive language. By means of comparative evaluation, we select the best performing for each of the three subtasks. Overall, we demonstrate that using only minimal preprocessing we are able to obtain competitive results.
    • RGCL at IDAT: deep learning models for irony detection in Arabic language

      Ranasinghe, Tharindu; Saadany, Hadeel; Plum, Alistair; Mandhari, Salim; Mohamed, Emad; Orasan, Constantin; Mitkov, Ruslan (IDAT, 2019-12-12)
      This article describes the system submitted by the RGCL team to the IDAT 2019 Shared Task: Irony Detection in Arabic Tweets. The system detects irony in Arabic tweets using deep learning. The paper evaluates the performance of several deep learning models, as well as how text cleaning and text pre-processing influence the accuracy of the system. Several runs were submitted. The highest F1 score achieved for one of the submissions was 0.818 making the team RGCL rank 4th out of 10 teams in final results. Overall, we present a system that uses minimal pre-processing but capable of achieving competitive results.
    • RGCL-WLV at SemEval-2019 Task 12: Toponym Detection

      Plum, Alistair; Ranasinghe, Tharindu; Calleja, Pablo; Orasan, Constantin; Mitkov, Ruslan (ACL, 2019-06-07)
      This article describes the system submitted by the RGCL-WLV team to the SemEval 2019 Task 12: Toponym resolution in scientific papers. The system detects toponyms using a bootstrapped machine learning (ML) approach which classifies names identified using gazetteers extracted from the GeoNames geographical database. The paper evaluates the performance of several ML classifiers, as well as how the gazetteers influence the accuracy of the system. Several runs were submitted. The highest precision achieved for one of the submissions was 89%, albeit it at a relatively low recall of 49%.
    • The rhetorical structure of science? A multidisciplinary analysis of article headings

      Thelwall, Mike (Elsevier, 2019-03-19)
      An effective structure helps an article to convey its core message. The optimal structure depends on the information to be conveyed and the expectations of the audience. In the current increasingly interdisciplinary era, structural norms can be confusing to the authors, reviewers and audiences of scientific articles. Despite this, no prior study has attempted to assess variations in the structure of academic papers across all disciplines. This article reports on the headings commonly used by over 1 million research articles from the PubMed Central Open Access collection, spanning 22 broad categories covering all academia and 172 out of 176 narrow categories. The results suggest that no headings are close to ubiquitous in any broad field and that there are substantial differences in the extent to which most headings are used. In the humanities, headings may be avoided altogether. Researchers should therefore be aware of unfamiliar structures that are nevertheless legitimate when reading, writing and reviewing articles.
    • The Role of Corpus Pattern Analysis in Machine Translation Evaluation

      Bechara, Hanna; Moze, Sara; El-Maarouf, Ismail; Orasan, Constantin; Hanks, Patrick; Mitkov, Ruslan (Tradulex, 2015)
      This paper takes a preliminary look at the relation between verb pattern matches in the Pattern Dictionary of English Verbs (PDEV) and translation quality through a qualitative analysis of human-ranked sentences from 5 different machine translation systems. The purpose of the analysis is not only to determine whether verbs in the automatic translations and their immediate contexts match any pre-existing semanto-syntactic pattern in PDEV, but also to establish links between hypothesis sentences and the verbs in the reference translation. It attempts to answer the question of whether or not the semantic and syntactic information captured by Corpus Pattern Analysis (CPA) can indicate whether a sentence is a “good” translation. Two human annotators manually identified the occurrence of patterns in 50 translations and indicated whether these patterns match any identified pattern in the corresponding reference translation. Results indicate that CPA can be used to distinguish between well and ill-formed sentences.
    • Scientific web intelligence: finding relationships in university webs

      Thelwall, Mike (ACM, 2005)
      Methods for analyzing university Web sites demonstrate strong patterns that can reveal interconnections between research fields.