Technology assisted research assessment: Algorithmic bias and transparency issues
AbstractPurpose: Technology is sometimes used to support assessments of academic research in the form of automatically generated bibliometrics for reviewers to consult during their evaluations or by replacing some or all human judgements. With Artificial Intelligence (AI), there is increasing scope to use technology to assist research assessment processes in new ways. Since transparency and fairness are widely considered important for research assessment and AI introduces new issues, this review investigates their implications. Design/methodology/approach: This articles reviews and briefly summarises transparency and fairness concerns in general terms and through the issues that they raise for various types of Technology Assisted Research Assessment (TARA). Findings: Whilst TARA can have varying levels of problems with both transparency and bias, in most contexts it is unclear whether it worsens the transparency and bias problems that are inherent in peer review. Originality: This is the first analysis that focuses on algorithmic bias and transparency issues for technology assisted research assessment.
CitationThelwall, M. and Kousha, K. (2023) Technology assisted research assessment: Algorithmic bias and transparency issues. Aslib Journal of Information Management, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/AJIM-04-2023-0119
JournalAslib Journal of Information Management
DescriptionThis is an author's accepted manuscript of an article published by Emerald on 02/10/2023 in Aslib Journal of Information Management, available online: https://doi.org/10.1108/AJIM-04-2023-0119 The accepted manuscript may differ from the final published version.
SponsorsThis study was funded by Research England, Scottish Funding Council, Higher Education Funding Council for Wales, and Department for the Economy, Northern Ireland as part of the Future Research Assessment Programme (https://www.jisc.ac.uk/future-research-assessment-programme).
Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by-nc/4.0/