Bickley, MatthewKousha, KayvanThelwall, MichaelWolfgang GlänzelSarah HeefferPei-Shan ChiRonald Rousseau2021-07-302021-07-302021-07-02Bickley, M., Kousha, K. & Thelwall, M. (2021) A systematic method for identifying references to academic research in grey literature in 18th International Conference on Scientometrics & Informetrics Proceedings, Wolfgang Glänzel, Sarah Heeffer, Pei-Shan Chi, Ronald Rousseau (eds.), ISSI, pp.121-132.97890803282282175-1935http://hdl.handle.net/2436/624237This is an accepted manuscript of an article published by International Society for Scientometrics and Informetrics in 18th INTERNATIONAL CONFERENCE ON SCIENTOMETRICS & INFORMETRICS Proceedings on 02/07/2021, available online: https://www.issi-society.org/proceedings/issi_2021/Proceedings%20ISSI%202021.pdf The accepted version of the publication may differ from the final published version.Grey literature is research that has not been written with the intent to publish in a traditional journal or book. From this, and due to its unstandardised nature, its impact in academia can be difficult to identify. Research impact can be assessed in multiple ways, with citation analysis a usual method. Impact can include the citing of an output, but in some situations, cited references may be useful in assessing ‘academic impact’. Cited references in grey literature, however, may reflect ‘non-academic impact’ of research such as in policy making, clinical practice or legislation. This study introduces and tests a semi-automatic method to measure cited references in grey literature with unknown standardisation of references. Metadata (lead author surname, title, year) of 2.45 million Russell Group university outputs were collected, added to known citation metadata from a 100-document sample of UK government grey literature, and then searched within each document, assessing the accuracy of 21 proposed variations of matching terms. A ‘best method’ is proposed (lead author surname and title in either order, maximum of 200 characters apart) to show cited references present, enabling the ability to analyse impact differences impact across subject areas and years within grey literature in future studies.application/pdfenGrey literatureGrey literature sourcesGrey literature impactCitation analysisUK governmentnon-academic impactNovel methodBland-AltmanBland-Altman analysisAltmetricsScientometricsInformetricsConference contribution2021-07-29