Show simple item record

dc.contributor.authorThelwall, Mike
dc.date.accessioned2022-02-14T09:16:57Z
dc.date.available2022-02-14T09:16:57Z
dc.date.issued2022-02-22
dc.identifier.citationThelwall, M. (2022) Can the quality of published academic journal articles be assessed with machine learning? Quantitative Science Studies, 3 (1), pp. 208–226.en
dc.identifier.issn2641-3337en
dc.identifier.doi10.1162/qss_a_00185
dc.identifier.urihttp://hdl.handle.net/2436/624598
dc.description© 2022 The Author. Published by MIT Press. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1162/qss_a_00185en
dc.description.abstractFormal assessments of the quality of the research produced by departments and universities are now conducted by many countries to monitor achievements and allocate performancerelated funding. These evaluations are hugely time consuming if conducted by postpublication peer review and are simplistic if based on citations or journal impact factors. This article investigates whether machine learning could help reduce the burden of peer review by using citations and metadata to learn how to score articles from a sample assessed by peer review. An experiment is used to underpin the discussion, attempting to predict journal citation thirds, as a proxy for article quality scores, for all Scopus narrow fields from 2014 to 2020. The results show that these proxy quality thirds can be predicted with above baseline accuracy in all 326 narrow fields, with Gradient Boosting Classifier, Random Forest Classifier, or Multinomial Naïve Bayes being the most accurate in nearly all cases. Nevertheless, the results partly leverage journal writing styles and topics, which are unwanted for some practical applications and cause substantial shifts in average scores between countries and between institutions within a country. There may be scope for predicting articles scores when the predictions have the highest probability.en
dc.formatapplication/pdfen
dc.language.isoenen
dc.publisherMIT Pressen
dc.relation.urlhttps://direct.mit.edu/qss/article/doi/10.1162/qss_a_00185/109627/Can-the-quality-of-published-academic-journalen
dc.subjectresearch evaluationen
dc.subjectmachine learningen
dc.subjectcitation analysisen
dc.subjecttext miningen
dc.titleCan the quality of published academic journal articles be assessed with machine learning?en
dc.typeJournal articleen
dc.identifier.journalQuantitative Science Studiesen
dc.date.updated2022-02-10T15:24:18Z
dc.date.accepted2022-02-08
rioxxterms.funderUK Research and Innovationen
rioxxterms.identifier.projectUOW14022022MTen
rioxxterms.versionAMen
rioxxterms.licenseref.urihttps://creativecommons.org/licenses/by/4.0/en
rioxxterms.licenseref.startdate2022-02-14en
dc.source.volume3
dc.source.issue1
dc.source.beginpage208
dc.source.endpage226
refterms.dateFCD2022-02-14T09:16:40Z
refterms.versionFCDAM
refterms.dateFOA2022-02-14T00:00:00Z


Files in this item

Thumbnail
Name:
qss_a_00185.pdf
Size:
1.036Mb
Format:
PDF

This item appears in the following Collection(s)

Show simple item record

https://creativecommons.org/licenses/by/4.0/
Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by/4.0/