Technology assisted research assessment: Algorithmic bias and transparency issues
Abstract
Purpose: Technology is sometimes used to support assessments of academic research in the form of automatically generated bibliometrics for reviewers to consult during their evaluations or by replacing some or all human judgements. With Artificial Intelligence (AI), there is increasing scope to use technology to assist research assessment processes in new ways. Since transparency and fairness are widely considered important for research assessment and AI introduces new issues, this review investigates their implications. Design/methodology/approach: This articles reviews and briefly summarises transparency and fairness concerns in general terms and through the issues that they raise for various types of Technology Assisted Research Assessment (TARA). Findings: Whilst TARA can have varying levels of problems with both transparency and bias, in most contexts it is unclear whether it worsens the transparency and bias problems that are inherent in peer review. Originality: This is the first analysis that focuses on algorithmic bias and transparency issues for technology assisted research assessment.Citation
Thelwall, M. and Kousha, K. (2023) Technology assisted research assessment: Algorithmic bias and transparency issues. Aslib Journal of Information Management, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/AJIM-04-2023-0119Publisher
EmeraldJournal
Aslib Journal of Information ManagementType
Journal articleLanguage
enDescription
This is an author's accepted manuscript of an article published by Emerald on 02/10/2023 in Aslib Journal of Information Management, available online: https://doi.org/10.1108/AJIM-04-2023-0119 The accepted manuscript may differ from the final published version.ISSN
2050-3806Sponsors
This study was funded by Research England, Scottish Funding Council, Higher Education Funding Council for Wales, and Department for the Economy, Northern Ireland as part of the Future Research Assessment Programme (https://www.jisc.ac.uk/future-research-assessment-programme).ae974a485f413a2113503eed53cd6c53
10.1108/AJIM-04-2023-0119
Scopus Count
Collections
Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by-nc/4.0/