How much are LLMs changing the language of academic papers?

This study investigates the influence of Large Language Models (LLMs) on academic publishing with a term frequency analysis of 12 LLM-associated terms in six major scholarly databases (Scopus, WoS, PubMed, Dimensions, OpenAlex, and PMC) from 2015 to 2024. From the proportion of articles containing them, all 12 LLM-associated terms had small increases in 2023 and large increases in 2024. For example, in 2024, underscore[s/d/ing] appeared in 20% of PMC open access publications, a fivefold increase from 4% in 2022, suggesting that LLMs had influenced the language of at least 16% of PMC documents in 2024. LLM-friendly terms like delve[s/d/ing] and underscore[s/d/ing] seem to have grown partly at the expense of equivalent more traditionally academic terms like investigate[s/d/ing] and highlight[s/ed/ing]. There were disciplinary differences between the 27 Scopus broad subject categories, with underscore[s/d/ing] being more common in Environmental Science and "delve" more frequently used in Business and Humanities. There were also differences in the terms found in different parts of papers. For example, unveil[s/ed/ing] was used particularly more frequently in titles in 2024 than 2022 (0.26% vs. 0.04%), whilst underscore[s/d/ing] was more prominent in abstracts (2.5% vs. 0.21%) in Scopus. The increases may be due mainly to the use of LLMs for translation and proof reading, but imitation by researchers may result in LLM-associated terms becoming a more organic part of future academic writing, unless there is a reaction against them. Finally, since 70% of Scopus papers acknowledging ChatGPT did not use any of the 12 terms in their titles or abstracts, the influence of LLMs is probably much wider.

Citation

Kousha, K. and Thelwall, M. (2025) How much are LLMs changing the language of academic papers? 20th International Conference on Scientometrics & Informetrics, Volume 2, pp. 915-927.

Type

Conference contribution

Language

en

Description

This is an author's accepted manuscript of a paper delivered at the 20th International conference on Scientometrics & Informetrics, June 23-27, 2025 Yerevan Armenia.

ISSN

2175-1935

ISBN

9789939120867

Collections

Faculty of Arts, Business and Social Sciences