Loading...
In which fields do higher impact journals publish higher quality articles?
Abdoli, Mahshid Levitt, Jonathan
Abdoli, Mahshid
Levitt, Jonathan
Editors
Other contributors
Affiliation
Epub Date
Issue Date
2023-05-18
Submitted date
Alternative
Abstract
The Journal Impact Factor (JIF) and other indicators that assess the average citation rate of articles in a journal are consulted by many academics and research evaluators, despite initiatives against overreliance on them. Undermining both practices, there is limited evidence about the extent to which journal impact indicators in any field relate to human judgements about the quality of the articles published in the field’s journals. In response, we compared average citation rates of journals against expert judgements of their articles in all fields of science. We used preliminary quality scores for 96,031 articles published 2014-18 from the UK Research Excellence Framework (REF) 2021. Unexpectedly, there was a positive correlation between expert judgements of article quality and average journal citation impact in all fields of science, although very weak in many fields and never strong. The strength of the correlation varied from 0.11 to 0.43 for the 27 broad fields of Scopus. The highest correlation for the 94 Scopus narrow fields with at least 750 articles was only 0.54, for Infectious Diseases, and there was only one negative correlation, for the mixed category Computer Science (all), probably due to the mixing. The average citation impact of a Scopus-indexed journal is therefore never completely irrelevant to the quality of an article but is also never a strong indicator of article quality. Since journal citation impact can at best moderately suggest article quality it should never be relied on for this, supporting the San Francisco Declaration on Research Assessment (DORA).
Citation
Thelwall, M., Kousha, K., Makita, M. et al. (2023) In which fields do higher impact journals publish higher quality articles?. Scientometrics 128, pp. 3915–3933. https://doi.org/10.1007/s11192-023-04735-0
Publisher
Journal
Research Unit
PubMed ID
PubMed Central ID
Embedded videos
Additional Links
Type
Journal article
Language
en
Description
This is an accepted manuscript of an article published by Springer in Scientometrics on 18/05/2023. The accepted version of the publication may differ from the final published version.
Series/Report no.
ISSN
0138-9130
EISSN
ISBN
ISMN
Gov't Doc #
Sponsors
This study was funded by Research England, Scottish Funding Council, Higher Education Funding Council for Wales, and Department for the Economy, Northern Ireland as part of the Future Research Assessment Programme (https://www.jisc.ac.uk/future-research-assessment-programme).