Loading...
Do altmetric scores reflect article quality? Evidence from the UK Research Excellence Framework 2021
Abdoli, Mahshid Levitt, Jonathan
Abdoli, Mahshid
Levitt, Jonathan
Editors
Other contributors
Affiliation
Epub Date
Issue Date
2023-03-14
Submitted date
Alternative
Abstract
Altmetrics are web-based quantitative impact or attention indicators for academic articles that have been proposed to supplement citation counts. This article reports the first assessment of the extent to which mature altmetrics from Altmetric.com and Mendeley associate with individual article quality scores. It exploits expert norm-referenced peer review scores from the UK Research Excellence Framework 2021 for 67,030+ journal articles in all fields 2014-17/18, split into 34 broadly field-based Units of Assessment (UoAs). Altmetrics correlated more strongly with research quality than previously found, although less strongly than raw and field normalised Scopus citation counts. Surprisingly, field normalising citation counts can reduce their strength as a quality indicator for articles in a single field. For most UoAs, Mendeley reader counts are the best altmetric (e.g., three Spearman correlations with quality scores above 0.5), tweet counts are also a moderate strength indicator in eight UoAs (Spearman correlations with quality scores above 0.3), ahead of news (8 correlations above 0.3, but generally weaker), blogs (5 correlations above 0.3), and Facebook (3 correlations above 0.3) citations, at least in the UK. In general, altmetrics are the strongest indicators of research quality in the health and physical sciences and weakest in the arts and humanities. Keywords: Altmetrics, Research Excellence Framework, REF2021, alternative indicators, scientometrics, bibliometrics, field normalised citations.
Citation
Thelwall, M., Kousha, K., Abdoli, M., Stuart, E., Makita, M., Wilson, P. and Levitt, J. (2023) Do altmetric scores reflect article quality? Evidence from the UK Research Excellence Framework 2021. Journal of the Association for Information Science and Technology, 2023, 74 (5), pp. 582– 593. https://doi.org/10.1002/asi.24751
Publisher
Research Unit
PubMed ID
PubMed Central ID
Embedded videos
Additional Links
Type
Journal article
Language
en
Description
© 2023 The Authors. Published by Wiley & Association for Information Science and Technology. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1002/asi.24751
Series/Report no.
ISSN
2330-1635
EISSN
ISBN
ISMN
Gov't Doc #
Sponsors
This study was funded by Research England, Scottish Funding Council, Higher Education Funding Council for Wales, and Department for the Economy, Northern Ireland.