Loading...
Does the perceived quality of interdisciplinary research vary between fields?
Abdoli, Mahshid Levitt, Jonathan
Abdoli, Mahshid
Levitt, Jonathan
Editors
Other contributors
Affiliation
Epub Date
Issue Date
2023-04-27
Submitted date
Files
Alternative
Abstract
Purpose: To assess whether interdisciplinary research evaluation scores vary between fields. Design/methodology/approach: We investigate whether published refereed journal articles were scored differently by expert assessors (two per output, agreeing a score and norm referencing) from multiple subject-based Units of Assessment (UoAs) in the REF2021 UK national research assessment exercise. The primary raw data was 8,015 journal articles published 2014-20 and evaluated by multiple UoAs, and the agreement rates were compared to the estimated agreement rates for articles multiply-evaluated within a single UoA. Findings: We estimated a 53% agreement rate on a four-point quality scale between UoAs for the same article and a within-UoA agreement rate of 70%. This suggests that quality scores vary more between fields than within fields for interdisciplinary research. There were also some hierarchies between fields, in the sense of UoAs that tended to give higher scores for the same article than others. Research limitations/implications: The results apply to one country and type of research evaluation. The agreement rate percentage estimates are both based on untested assumptions about the extent of cross-checking scores for the same articles in the REF, so the inferences about the agreement rates are tenuous. Practical implications: The results underline the importance of choosing relevant fields for any type of research evaluation. Originality: This is the first evaluation of the extent to which a careful peer review exercise generates different scores for the same articles between disciplines.
Citation
Thelwall, M., Kousha, K., Stuart, E., Makita, M., Abdoli, M., Wilson, P. and Levitt, J.M. (2023), "Does the perceived quality of interdisciplinary research vary between fields?", Journal of Documentation, Vol. 79 No. 6, pp. 1514-1531. https://doi.org/10.1108/JD-01-2023-0012
Publisher
Journal
Research Unit
PubMed ID
PubMed Central ID
Embedded videos
Additional Links
Type
Journal article
Language
en
Description
This is an accepted manuscript of an article published in Journal of Documentation by Emerald on 27/04/2023, available online: 10.1108/JD-01-2023-0012 The accepted version of the publication may differ from the final published version.
Series/Report no.
ISSN
0022-0418
EISSN
ISBN
ISMN
Gov't Doc #
Sponsors
This study was funded by Research England, Scottish Funding Council, Higher Education Funding Council for Wales, and Department for the Economy, Northern Ireland as part of the Future Research Assessment Programme (https://www.jisc.ac.uk/future-research-assessment-programme).