• All downhill from the PhD? The typical impact trajectory of US academic careers

      Thelwall, Michael; Fairclough, Ruth (MIT Press, 2020-09-04)
      Within academia, mature researchers tend to be more senior, but do they also tend to write higher impact articles? This article assesses long-term publishing (16+ years) United States (US) researchers, contrasting them with shorter-term publishing researchers (1, 6 or 10 years). A long-term US researcher is operationalised as having a first Scopus-indexed journal article in exactly 2001 and one in 2016-2019, with US main affiliations in their first and last articles. Researchers publishing in large teams (11+ authors) were excluded. The average field and year normalised citation impact of long- and shorter-term US researchers’ journal articles decreases over time relative to the national average, with especially large falls to the last articles published that may be at least partly due to a decline in self-citations. In many cases researchers start by publishing above US average citation impact research and end by publishing below US average citation impact research. Thus, research managers should not assume that senior researchers will usually write the highest impact papers.
    • A Bayesian hurdle quantile regression model for citation analysis with mass points at lower values

      Shahmandi, Marzieh; Wilson, Paul; Thelwall, Michael (MIT Press, 2021-07-21)
      Quantile regression is a technique to analyse the effects of a set of independent variables on the entire distribution of a continuous response variable. Quantile regression presents a complete picture of the effects on the location, scale, and shape of the dependent variable at all points, not just at the mean. This research focuses on two challenges for the analysis of citation counts by quantile regression: discontinuity and substantial mass points at lower counts, such as zero, one, two, and three. A Bayesian two-part hurdle quantile regression model was proposed by King and Song (2019) as a suitable candidate for modeling count data with a substantial mass point at zero. Their model allows the zeros and non-zeros to be modeled independently but simultaneously. It uses quantile regression for modeling the nonzero data and logistic regression for modeling the probability of zeros versus nonzeros. Nevertheless, the current paper shows that substantial mass points also at one, two, and three for citation counts will nearly certainly affect the estimation of parameters in the quantile regression part of the model in a similar manner to the mass point at zero. We update the King and Song model by shifting the hurdle point from zero to three, past the main mass points. The new model delivers more accurate quantile regression for moderately to highly cited articles, especially at quantiles corresponding to values just beyond the mass points, and enables estimates of the extent to which factors influence the chances that an article will be low cited. To illustrate the advantage and potential of this method, it is applied separately to both simulated citation counts and also seven Scopus fields with collaboration, title length, and journal internationality as independent variables.
    • Coronavirus research before 2020 is more relevant than ever, especially when interpreted for COVID-19

      Thelwall, Michael (MIT Press, 2020-12-01)
      The speed with which biomedical specialists were able to identify and characterise COVID-19 was partly due to prior research with other coronaviruses. Early epidemiological comparisons with Severe Acute Respiratory Syndrome (SARS) and Middle East Respiratory Syndrome (MERS), also made it easier to predict COVID-19’s likely spread and lethality. This article assesses whether academic interest in prior coronavirus research has translated into interest in the primary source material, using Mendeley reader counts for early academic impact evidence. The results confirm that SARS and MERS research 2008-2017 experienced anomalously high increases in Mendeley readers in April-May 2020. Nevertheless, studies learning COVID-19 lessons from SARS and MERS or using them as a benchmark for COVID-19 have generated much more academic interest than primary studies of SARS or MERS. Thus, research that interprets prior relevant research for new diseases when they are discovered seems to be particularly important to help researchers to understand its implications in the new context.
    • COVID-19 publications: Database coverage, citations, readers, tweets, news, Facebook walls, Reddit posts

      Kousha, Kayvan; Thelwall, Michael (MIT Press, 2020-09-04)
      The COVID-19 pandemic requires a fast response from researchers to help address biological, medical and public health issues to minimize its impact. In this rapidly evolving context, scholars, professionals and the public may need to quickly identify important new studies. In response, this paper assesses the coverage of scholarly databases and impact indicators during 21 March to 18 April 2020. The rapidly increasing volume of research, is particularly accessible through Dimensions, and less through Scopus, the Web of Science, and PubMed. Google Scholar’s results included many false matches. A few COVID-19 papers from the 21,395 in Dimensions were already highly cited, with substantial news and social media attention. For this topic, in contrast to previous studies, there seems to be a high degree of convergence between articles shared in the social web and citation counts, at least in the short term. In particular, articles that are extensively tweeted on the day first indexed are likely to be highly read and relatively highly cited three weeks later. Researchers needing wide scope literature searches (rather than health focused PubMed or medRxiv searches) should start with Dimensions (or Google Scholar) and can use tweet and Mendeley reader counts as indicators of likely importance.
    • Domestic researchers with longer careers generate higher average citation impact but it does not increase over time

      Maflahi, Nabeil; Thelwall, Michael (MIT Press, 2021-07-15)
      Information about the relative strengths of scholars is needed for the efficient running of knowledge systems. Since academic research requires many skills, more experienced researchers might produce better research and attract more citations. This article assesses career citation impact changes 2001-2016 for domestic researchers (definition: first and last Scopus journal article in the same country) from the twelve nations with most Scopus documents. Careers are analysed longitudinally, so that changes are not due to personnel evolution, such as researchers leaving or entering a country. The results show that long term domestic researchers do not tend to improve their citation impact over time but tend to achieve their average citation impact by their first or second Scopus journal article. In some countries, this citation impact subsequently declines. These longer-term domestic researchers have higher citation impact than the national average in all countries, however, whereas scholars publishing only one journal article have substantially lower citation impact in all countries. The results are consistent with an efficiently functioning researcher selection system but cast slight doubt on the long-term citation impact potential of long-term domestic researchers. Research and funding policies may need to accommodate these patterns when citation impact is a relevant indicator.
    • Gender differences in citation impact for 27 fields and 6 English speaking countries 1996-2014

      Thelwall, Michael (MIT Press, 2020-06-25)
      Initiatives addressing the lack of women in many academic fields, and the general lack of senior women, need to be informed about causes of any gender differences that may affect career progression, including citation impact. Previous research about gender differences in journal article citation impact has found the direction of any difference to vary by country and field but has usually avoided discussions of the magnitude and wider significance of any differences and has not been systematic in terms of fields and/or time. This study investigates differences in citation impact between male and female first-authored research for 27 broad fields and 6 large English-speaking countries (Australia, Canada, Ireland, New Zealand, UK and USA) 1996-2014. The results show an overall female first author citation advantage, although in most broad fields it is reversed in all countries for some years. International differences include Medicine having a female first author citation advantage for all years in Australia but a male citation advantage for most years in Canada. There was no general trend for the gender difference to increase or decrease over time. The average effect size is small, however, and unlikely to have a substantial influence on overall gender differences in researcher careers.
    • A gender equality paradox in academic publishing: Countries with a higher proportion of female first-authored journal articles have larger first author gender disparities between fields

      Thelwall, Michael; Mas-Bleda, Amalia (MIT Press, 2020-05-28)
      Current attempts to address the shortfall of female researchers in Science, Technology, Engineering and Mathematics (STEM) have not yet succeeded despite other academic subjects having female majorities. This article investigates the extent to which gender disparities are subject-wide or nation-specific by a first author gender comparison of 30 million articles from all 27 Scopus broad fields within the 31 countries with the most Scopus-indexed articles 2014-18. The results show overall and geocultural patterns as well as individual national differences. Almost half of the subjects were always more male (7; e.g., Mathematics) or always more female (6; e.g., Immunology & Microbiology) than the national average. A strong overall trend (Spearman correlation 0.546) is for countries with a higher proportion of female first-authored research to also have larger differences in gender disparities between fields (correlation 0.314 for gender ratios). This confirms the international gender equality paradox previously found for degree subject choices: increased gender equality overall associates with moderately greater gender differentiation between subjects. This is consistent with previous USA-based claims that gender differences in academic careers are partly due to (socially constrained) gender differences in personal preferences. Radical solutions may therefore be needed for some STEM subjects to overcome gender disparities.
    • Google Books, Scopus, Microsoft Academic and Mendeley for impact assessment of doctoral dissertations: A multidisciplinary analysis of the UK

      Kousha, Kayvan; Thelwall, Mike (MIT Press, 2020-06-25)
      A research doctorate normally culminates in publishing a dissertation reporting a substantial body of novel work. In the absence of a suitable citation index, this article explores the relative merits of alternative methods for the large-scale assessment of dissertation impact, using 150,740 UK doctoral dissertations from 2009-2018. Systematic methods for this were designed for Google Books, Scopus, Microsoft Academic, and Mendeley. Less than 1 in 8 UK doctoral dissertations had at least one Scopus (12%), Microsoft Academic (11%) or Google Books citation (9%), or at least one Mendeley reader (5%). These percentages varied substantially by subject area and publication year. Google Books citations were more common in the Arts and Humanities (18%), whereas Scopus and Microsoft Academic citations were more numerous in Engineering (24%). In the Social Sciences, Google Books (13%) and Scopus (12%) citations were important and in Medical Sciences, Scopus and Microsoft Academic citations to dissertations were rare (6%). Few dissertations had Mendeley readers (from 3% in Science to 8% in the Social Sciences) and further analysis suggests that Google Scholar finds more citations but does not report information about all dissertations within a repository and is not a practical tool for large-scale impact assessment.
    • Greater female first author citation advantages do not associate with reduced or reducing gender disparities in academia

      Thelwall, Michael; Sud, Pardeep (MIT Press, 2020-09-04)
      Ongoing problems attracting women into many Science, Technology, Engineering and Mathematics (STEM) subjects have many potential explanations. This article investigates whether possible under-citation of women associates with lower proportions of, or increases in, women in a subject. It uses six million articles published 1996-2012 across up to 331 fields in six mainly English-speaking countries: Australia, Canada, Ireland, New Zealand, UK and USA. The proportion of female first and last authored articles in each year was calculated and 4968 regressions were run to detect first author gender advantages in field normalised article citations. The proportion of female first authors in each field correlated highly between countries and the female first author citation advantages derived from the regressions correlated moderately to strongly between countries, so both are relatively field-specific. There was a weak tendency in the USA and New Zealand for female citation advantages to be stronger in fields with fewer women, after excluding small fields, but no other association evidence. There was no evidence of female citation advantages or disadvantages to be a cause or effect of changes in the proportions of women in a field for any country. Inappropriate uses of career-level citations are a likelier source of gender inequities.
    • How common are explicit research questions in journal articles?

      Thelwall, Michael; Mas-Bleda, Amalia (MIT Press, 2020-12-01)
      Although explicitly labelled research questions seem to be central to some fields, others do not need them. This may confuse authors, editors, readers and reviewers of multidisciplinary research. This article assesses the extent to which research questions are explicitly mentioned in 17 out of 22 areas of scholarship from 2000 to 2018 by searching over a million full-text open access journal articles. Research questions were almost never explicitly mentioned (under 2%) by articles in engineering, physical, life and medical sciences, and were the exception (always under 20%) for the broad fields in which they were least rare: computing, philosophy, theology and social sciences. Nevertheless, research questions were increasingly mentioned explicitly in all fields investigated, despite a rate of 1.8% overall (1.1% after correcting for irrelevant matches). Other terminology for an article’s purpose may be more widely used instead, including aims, objectives, goals, hypotheses, and purposes, although no terminology occurs in a majority of articles in any broad field tested. Authors, editors, readers and reviewers should therefore be aware that the use of explicitly labelled research questions or other explicit research purpose terminology is non-standard in most or all broad fields, although it is becoming less rare. Keywords: Research purpose statements; research article structures; research questions; research aims; research goals.
    • Large publishing consortia produce higher citation impact research but co-author contributions are hard to evaluate

      Thelwall, Michael (MIT Press, 2020-02-20)
      This paper introduces a simple agglomerative clustering method to identify large publishing consortia with at least 20 authors and 80% shared authorship between articles. Based on Scopus journal articles 1996-2018, under these criteria, nearly all (88%) of the large consortia published research with citation impact above the world average, with the exceptions being mainly the newer consortia for which average citation counts are unreliable. On average, consortium research had almost double (1.95) the world average citation impact on the log scale used (Mean Normalised Log Citation Score). At least partial alphabetical author ordering was the norm in most consortia. The 250 largest consortia were for nuclear physics and astronomy around expensive equipment, and for predominantly health-related issues in genomics, medicine, public health, microbiology and neuropsychology. For the health-related issues, except for the first and last few authors, authorship seem to primary indicate contributions to the shared project infrastructure necessary to gather the raw data. It is impossible for research evaluators to identify the contributions of individual authors in the huge alphabetical consortia of physics and astronomy, and problematic for the middle and end authors of health-related consortia. For small scale evaluations, authorship contribution statements could be used, when available.
    • Mendeley reader counts for US computer science conference papers and journal articles

      Thelwall, Mike (MIT Press, 2019-12-31)
      Although bibliometrics are normally applied to journal articles when used to support research evaluations, conference papers are at least as important in fast-moving computingrelated fields. It is therefore important to assess the relative advantages of citations and altmetrics for computing conference papers to make an informed decision about which, if any, to use. This paper compares Scopus citations with Mendeley reader counts for conference papers and journal articles that were published between 1996 and 2018 in 11 computing fields and had at least one US author. The data showed high correlations between Scopus citation counts and Mendeley reader counts in all fields and most years, but with few Mendeley readers for older conference papers and few Scopus citations for new conference papers and journal articles. The results therefore suggest that Mendeley reader counts have a substantial advantage over citation counts for recently-published conference papers due to their greater speed, but are unsuitable for older conference papers.
    • Pot, kettle: Nonliteral titles aren’t (natural) science

      Thelwall, Michael (MIT Press, 2020-06-09)
      Researchers may be tempted to attract attention through poetic titles for their publications, but would this be mistaken in some fields? Whilst poetic titles are known to be common in medicine, it is not clear whether the practice is widespread elsewhere. This article investigates the prevalence of poetic expressions in journal article titles 1996-2019 in 3.3 million articles from all 27 Scopus broad fields. Expressions were identified by manually checking all phrases with at least 5 words that occurred at least 25 times, finding 149 stock phrases, idioms, sayings, literary allusions, film names and song titles or lyrics. The expressions found are most common in the social sciences and the humanities. They are also relatively common in medicine, but almost absent from engineering and the natural and formal sciences. The differences may reflect the less hierarchical and more varied nature of the social sciences and humanities, where interesting titles may attract an audience. In engineering, natural science and formal science fields, authors should take extra care with poetic expressions, in case their choice is judged inappropriate. This includes interdisciplinary research overlapping these areas. Conversely, reviewers of interdisciplinary research involving the social sciences should be more tolerant of poetic license
    • Which aspects of the open science agenda are most relevant to scientometric research and publishing? An opinion paper

      Bornmann, Lutz; Guns, Raf; Thelwall, Michael; Wolfram, Dietmar (MIT Press, 2021-02-10)
      Open Science is an umbrella term that encompasses many recommendations for possible changes in research practices, management, and publishing with the objective to increase transparency and accessibility. This has become an important science policy issue that all disciplines should consider. Many Open Science recommendations may be valuable for the further development of research and publishing but not all are relevant to all fields. This opinion paper considers the aspects of Open Science that are most relevant for scientometricians, discussing how they can be usefully applied.
    • Which types of online evidence show the non-academic benefits of research? Websites cited in UK impact case studies

      Kousha, Kayvan; Thelwall, Mike; Abdoli, Mahshid (MIT press, 2021-06-30)
      Whilst funders increasingly request evidence of the societal benefits of research, all academics in the UK must periodically provide this information to gain part of their block funding within the Research Excellence Framework (REF). The impact case studies produced in the UK are public and can therefore be used to gain insights into the types of sources used to justify societal impact claims. This study focuses on the URLs cited as evidence in the last public REF to help researchers and resource providers to understand what types can be used and the disciplinary differences in their uptake. Based on a new semi-automatic method to classify the URLs cited in impact case studies, the results show that there are a few key online types of source for most broad fields, but these sources differ substantially between subject areas. For example, news websites are more important in some fields than others, and YouTube is sometimes used for multimedia evidence in the arts and humanities. Knowledge of the common sources selected independently by thousands of researchers may help others to identify suitable sources for the complex task of evidencing societal impacts.