In which fields do higher impact journals publish higher quality articles?
Authors
Thelwall, MikeKousha, Kayvan
Makita, Meiko
Abdoli, Mahshid
Stuart, Emma
Wilson, Paul
Levitt, Jonathan
Issue Date
2023-05-18
Metadata
Show full item recordAbstract
The Journal Impact Factor (JIF) and other indicators that assess the average citation rate of articles in a journal are consulted by many academics and research evaluators, despite initiatives against overreliance on them. Undermining both practices, there is limited evidence about the extent to which journal impact indicators in any field relate to human judgements about the quality of the articles published in the field’s journals. In response, we compared average citation rates of journals against expert judgements of their articles in all fields of science. We used preliminary quality scores for 96,031 articles published 2014-18 from the UK Research Excellence Framework (REF) 2021. Unexpectedly, there was a positive correlation between expert judgements of article quality and average journal citation impact in all fields of science, although very weak in many fields and never strong. The strength of the correlation varied from 0.11 to 0.43 for the 27 broad fields of Scopus. The highest correlation for the 94 Scopus narrow fields with at least 750 articles was only 0.54, for Infectious Diseases, and there was only one negative correlation, for the mixed category Computer Science (all), probably due to the mixing. The average citation impact of a Scopus-indexed journal is therefore never completely irrelevant to the quality of an article but is also never a strong indicator of article quality. Since journal citation impact can at best moderately suggest article quality it should never be relied on for this, supporting the San Francisco Declaration on Research Assessment (DORA).Citation
Thelwall, M., Kousha, K., Makita, M. et al. (2023) In which fields do higher impact journals publish higher quality articles?. Scientometrics 128, pp. 3915–3933. https://doi.org/10.1007/s11192-023-04735-0Publisher
SpringerJournal
ScientometricsAdditional Links
https://www.springer.com/journal/11192Type
Journal articleLanguage
enDescription
This is an accepted manuscript of an article published by Springer in Scientometrics on 18/05/2023. The accepted version of the publication may differ from the final published version.ISSN
0138-9130Sponsors
This study was funded by Research England, Scottish Funding Council, Higher Education Funding Council for Wales, and Department for the Economy, Northern Ireland as part of the Future Research Assessment Programme (https://www.jisc.ac.uk/future-research-assessment-programme).ae974a485f413a2113503eed53cd6c53
10.1007/s11192-023-04735-0