A recent study led by four researchers from the University of Tübingen and Northwestern University in Germany has revealed that the use of Large Language Models (LLMs) has a significant impact on scientific writing. Using overword analysis methods, the study highlights a sharp increase in the use of certain words since the introduction of LLMs at the end of 2022.
The study was conducted by analyzing more than 14 million abstracts published in PubMed between 2010 and 2024. The researchers compared the relative frequency of words before and after the LLM era to identify changes in lexical selection. Results showed that many words that were rarely used before, such as “delves,” “showcasing,” and “underscorres,” have seen a surge in usage since LLM became more commonly used.
Dr. Andreas Müller, one of the lead researchers from the University of Tübingen, explained that this increase indicates the use of LLMs in the writing process of scientific paper abstracts. “We found that at least 10% of abstracts published in 2024 have LLMs in the writing process,” he said. These findings highlight the importance of detecting the use of LLMs, because the resulting texts may appear human but contain inaccurate references or false claims.
The study also compared the post-LLM word use spike to word spikes during major global health events, such as the COVID-19 pandemic. Dr. Muller explained that prior to the LLM era, word spikes were typically associated with major global events, such as the Ebola outbreak in 2015 and the COVID-19 pandemic of 2020-2022. However, post-LLM word spikes tend to be concentrated in style words, such as vocabulary words, word of nature, and word of statement.
While an increase in the use of these words is a natural part of the evolution of a language, the researchers emphasize that this sudden and significant increase was rarely seen before the LLM era. They also point out that LLM use may be more common among non-native writers who need help editing their English writing.
This discovery paves the way for improving humans' ability to detect and remove unnatural style words from text generated by LLMs. The researchers hope that knowledge of LLM marker words will enable human editors to more effectively filter generated text before disseminating it to the global scientific community.
The study was pre-published earlier this month and is expected to spur further discussion about the impact of AI-generative technologies on contemporary scientific communication.
Tags: kecerdasan buatan
