A systematic method for identifying references to academic research in grey literature
Abstract
Grey literature encompasses documents not published in academic journals or books. Some grey literature has substantial societal importance, such as medical guidelines, government analyses and pressure group reports. Academic research cited in such documents may therefore have had indirect societal impact, such as in policy making, clinical practice or legislation. Identifying citations to academic research from grey literature may therefore help assess its societal impacts. This is difficult, however, due to the variety of document and referencing formats used in grey literature, even from a single organisation. In response, this study introduces and tests a semi-automatic method to match academic journal articles with unstandardised grey literature cited references. For this, the metadata (lead author last name, title, year) of 2.45 million UK Russell Group university outputs was matched against a 100-document sample of UK government grey literature to assess the accuracy of 21 matching heuristics. The optimal method (lead author last name and title in either order, maximum of 200 characters apart) is sufficiently accurate and scalable to make the task of matching research outputs to grey literature references feasible. The method was then applied to 3347 government publications, showing approximately 23% of UK government grey literature in this study contained at least one reference to UK Russell Group university output, and of this grey literature, an average of 3.79 references were present per document. The applied method also shows that economics and environmental science academic research is most cited between 2010 and 2018.Citation
Bickley, M.S., Kousha, K. & Thelwall, M. (2022) A systematic method for identifying references to academic research in grey literature. Scientometrics, 127, pp.6913–6933. https://doi.org/10.1007/s11192-022-04408-4Publisher
SpringerJournal
ScientometricsAdditional Links
https://link.springer.com/article/10.1007/s11192-022-04408-4Type
Journal articleLanguage
enDescription
This is an accepted manuscript of an article published by Springer on 23/06/2022, available online: https://doi.org/10.1007/s11192-022-04408-4 The accepted version of the publication may differ from the final published version. For re-use please see the publisher's terms and conditions.ISSN
0138-9130ae974a485f413a2113503eed53cd6c53
10.1007/s11192-022-04408-4