At least 13% of the research summaries published in 2024 contain the word “style” considered favored by these AI systems, suggesting an analysis of over 15 million biomedical papers published between 2010 and 2024, and thus may have gained help from a large-scale linguistic model.
Large language models with artificial intelligence are trained with vast amounts of text, allowing them to meet the human needs of natural language.
Researchers at the University of Tübingen in Germany said that AI models have caused dramatic changes in the vocabulary used in academic writing, and that speculations about the impact in scientific writing are common.
The study, published in Journal Science, revealed that the emergence of large-scale language models, including “DELVES”, “showcasing”, “Underscore”, “Potential”, “Survey Results” and “Important”, led to an increase in the use of certain “stylistic words.”
The authors explained that the word changes used between 2023 and 2024 are not “content-related nouns,” but rather verbs and adjectives that influence the styles preferred by large-scale linguistic models.
For analysis, researchers used a general public health approach during the Covid-19 pandemic to estimate excess deaths. This method evaluates the impact of Covid-19 on mortality rates to compare deaths during the pandemic with previous deaths.
The modified approach for this analysis was called by researchers the “excessive word” framework.
The findings show the “unprecedented impact” of AI models on scientific writing in biomedical research.
“We studied vocabulary changes in biomedical summaries indexed by PubMed from 2010 to 2024, showing that the appearance of the (big language model) led to a rapid increase in the frequency of words of a particular style,” the author writes.
PubMed is a search engine that provides access to biomedical and life science literature published from around the world.
“This excess word analysis suggests that at least 13.5% of the 2024 summary was processed by (a large-scale language model),” the team wrote.
The figures were found to differ between sectors, countries and magazines, and in some cases reached 40%.
In the field of biomedical research, approximately 20% of abstracts were involved in the use of large-scale language models. The researchers said the reason is that computer science researchers are tech-savvy and willing to adopt.
In English-speaking countries, AI systems help authors edit English texts.
However, they added that factors such as short publication timelines in the field of computational should enable these journals to detect AI use early.
Therefore, the findings of the study could be re-evaluated after several publication cycles in all disciplines and journals where the methods used here are useful, the team said.
