Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? Evidence from the F1000Research post-publication open peer review publishing model
Abstract
As part of moves towards open knowledge practices, making peer review open is cited as a way to enable fuller scrutiny and transparency of assessments around research. There are now many flavours of open peer review in use across scholarly publishing, including where reviews are fully attributable and the reviewer is named. This study examines whether there is any evidence of bias in two areas of common critique of open, non-anonymous (named) peer review – and used in the post-publication, peer review system operated by the open-access scholarly publishing platform F1000Research. First, is there evidence of potential bias where a reviewer based in a specific country assesses the work of an author also based in the same country? Second, are reviewers influenced by being able to see the comments and know the origins of a previous reviewer? Based on over 4 years’ of open peer review data, we found some evidence, albeit weak, that being based in the same country as an author may influence a reviewer’s decision, while there was insufficient evidence to conclude that being able to read an existing published review prior to submitting their review encourages conformity. Thus, whilst immediate publishing of peer review reports appears to be unproblematic, caution may be needed when selecting same-country reviewers in open systems if other studies confirm these results.Citation
Allen, L., Papas, E., Nyakoojo, Z., Weigert, V. and Thelwall, M (2020) Does the use of open, non-anonymous peer review in scholarly publishing introduce bias? Evidence from the F1000Research post-publication open peer review publishing model, Journal of Information Science, 47 (6), pp. 809-820. https://doi.org/10.1177/0165551520938678Publisher
SAGEJournal
Journal of Information ScienceAdditional Links
https://journals.sagepub.com/doi/10.1177/0165551520938678Type
Journal articleLanguage
enDescription
This is an accepted manuscript of an article published by SAGE in Journal of Information Science on 05/07/2020. The published version can be accessed here: https://doi.org/10.1177/0165551520938678 The accepted version of the publication may differ from the final published version.ISSN
0165-5515ae974a485f413a2113503eed53cd6c53
10.1177/0165551520938678
Scopus Count
Collections
Except where otherwise noted, this item's license is described as https://creativecommons.org/licenses/by-nc-nd/4.0/
Related items
Showing items related by title, author, creator and subject.
-
Artificial intelligence to support publishing and peer review: A summary and reviewKousha, Kayvan; Thelwall, Mike (Wiley, 2023-08-08)Technology is being developed to support the peer review processes of journals, conferences, funders, universities, and national research evaluations. This literature and software summary discusses the partial or complete automation of several publishing-related tasks: suggesting appropriate journals for an article, providing quality control for submitted papers, finding suitable reviewers for submitted papers or grant proposals, reviewing, and review evaluation. It also discusses attempts to estimate article quality from peer review text and scores as well as from post-publication scores but not from bibliometric data. The literature and existing examples of working technology show that automation is useful for helping to find reviewers and there is good evidence that it can sometimes help with initial quality control of submitted manuscripts. Much other software supporting publishing and editorial work exists and is being used, but without published academic evaluations of its efficacy. The value of artificial intelligence (AI) to support reviewing has not been clearly demonstrated yet, however. Finally, whilst peer review text and scores can theoretically have value for post-publication research assessment, it is not yet widely enough available to be a practical evidence source for systematic automation.
-
Examining the effects of sport and exercise interventions on body image among adolescent girls: A systematic reviewMcIntosh-Dalmedo, Sharon; Nicholls, Wendy; Devonport, Tracey; Friesen, Andrew P (University of South Alabama, 2018-09-01)Body image dissatisfaction among females is suggested to be so widespread, that is has been described as normative discontent. Consequently, there is great interest in the development of interventions that may enhance body image perceptions. The aim of the present systematic review was to investigate the effects of sport and exercise interventions on body image among adolescent females. Following preferred reporting items for systematic reviews and meta-analyses guidelines (Higgins & Green, 2009; Petticrew & Roberts, 2005), a search of six electronic databases produced 4,210 records of which six met the inclusion criteria. The methodological quality of included articles was assessed using the Standard Quality Assessment (Kmet, Lee, & Cook, 2004). This yielded a mean score for quality of .90 (SD = 0.22), indicating poor quality of research. In two studies, significant and positive change was observed in body image following intervention (aerobics or self-selected sports activities) in comparison to a control condition. In four studies, no significant effect of intervention on body image was observed. We conclude that there is insufficient evidence to suggest that sport and exercise interventions can improve body image. Furthermore, due to the limitations of existing research highlighted within this review, findings suggesting positive influence should be interpreted with caution. Recommendations for improving the methodological quality of research examining the influence of sport and exercise interventions on body image are proposed. This includes considerations such as participant sampling, control conditions/groups, measurement of key variables, intervention features, and analysis of data.
-
A systematic review of the association between emotions and eating behaviour in normal and overweight adult populationsDevonport, Tracey; Nicholls, Wendy; Fullerton, Christopher L. (Sage Publications LTD, 2017-03-20)A systematic review was completed according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. A comprehensive search of four electronic databases (2004-2015) yielded 60017 articles, of which 29 met inclusion criteria. Included studies performed poorly on data quality analysis in terms of randomization and controlling for confounding factors. Participant’s BMI scores range from 19.73 (SD = 1.54) to 28.4 (SD = 1.4) kg/m2. Where positive and negative affect were compared, food was more likely to be consumed in response to positive affect. With regards to discrete emotions; stress, depression, and sadness consistently elicited eating behaviours that fall outside of nutritional recommendations (e.g., increased food intake, poor nutritional food choices). The role of moderators including individual differences in dietary restraint and emotional eating, as well as methodological considerations, such as means of eliciting and measuring emotions, may account for equivocality with regards to some emotion and eating associations. The paper concludes with recommendations for future research and implications for practice.