Librarian Andrew Gray made the “very surprising” discovery. He analyzed 5 million scientific studies published last year and ranked them as meticulous (up 137%), complex (117%), admirable (83%), and meticulous (up 137%). (59%) found a sudden increase in the use of certain words. A librarian at University College London has found only one explanation for this increase. That means tens of thousands of researchers are writing, or at least “refining” their research using ChatGPT (or other similar large-scale language modeling tools with artificial intelligence).
There are blatant examples. A team of Chinese scientists published their research on lithium batteries on February 17th. The study, published in the journal Elsevier Publishers, begins as follows. A promising candidate for… ” The author apparently asked ChatGPT for an introduction and accidentally copied it verbatim. Another Elsevier paper published on March 8 by Israeli researchers includes the following sentence: I am her AI language model, so certain data. ” Then, a few months ago, three Chinese scientists unveiled a bizarre drawing of a giant penis-like rat, generated by artificial intelligence to study sperm precursor cells.
Andrew Gray says at least 60,000 scientific studies (more than 1% of studies analyzed in 2023) were written using ChatGPT (a tool launched in late 2022) or similar tools. We estimate that. “I think the extreme case of writing an entire study using ChatGPT is rare,” says Gray, 41, a librarian in Scotland. In his opinion, most of the time artificial intelligence is properly used to “polish” texts, i.e. to identify typos or facilitate translation into English, but there are large gray areas. , and some scientists are further aiding ChatGPT without verifying that part. result. “At the moment, it is impossible to know how large this gray area is. Scientific journals do not require authors to declare their use of ChatGPT, so there is little transparency,” he said. laments.
As James Zou's team at Stanford University demonstrated, artificial intelligence language models overuse certain words. These tend to be terms with positive connotations, such as admirable, meticulous, complex, innovative, and versatile. In March, Zou and his colleagues warned that reviewers of scientific research themselves were using these programs to write their reviews before publication. A group at Stanford University analyzed peer reviews of research presented at two international artificial intelligence conferences and found that the odds of the word “meticulous” appearing increased by 35 times.
Meanwhile, Zou's team failed to detect any significant traces of ChatGPT in the modifications published in prestigious journals. Nature group. ChatGPT use was associated with lower peer review quality. “I think that's really concerning,” Gray explains. “If we know that using these tools to write reviews results in poor quality results, we should reflect on how the tools are being used to write research and what that means. “We need to do that,” says a librarian at University College London.A year after ChatGPT was launched, one in three scientists admitted to using the tool to write research, according to the magazine's research. Nature.
According to Gray's analysis, the word “complex” appeared in 109,000 studies in 2023, more than double the previous year's average of 50,000. The term “meticulously” increased from about 12,300 studies in 2022 to more than 28,000 in 2023. Meanwhile, examples of “worthy of praise” increased from 6,500 to nearly 12,000. The researcher jokes that his colleagues have complimented the thoroughness of his report, but a draft is yet to be published in a professional journal.
Few studies report whether they used artificial intelligence. Gray warned of the danger of a “vicious cycle” in which subsequent versions of ChatGPT are trained on scientific papers written by older versions, creating increasingly admirable research that is complex, thorough, and above all, devoid of substance. Masu.
Documentation professor Ángel María Delgado Vázquez emphasizes that the new analysis focuses on the study of English. Delgado Vázquez, a researcher at Pablo de Olavide University in Seville, Spain, says, “Researchers who don't speak English as their first language often use ChatGPT to help them write and improve their English.'' “In my environment, people primarily use his ChatGPT for initial translations or to save those translations directly,” he says. A Spanish professor said he would like to see an analysis of the origins of authors who use unusual terms.
Another favorite word for AI is “dig”. Jeremy Nguyen, a researcher at Swinburne University of Technology in Australia, calculated that the word “investigate” is used in the following languages: 0.5% of medical researchBefore ChatGPT it was less than 0.04%. thousands of researchers Suddenly I'm digging.
Librarian Andrew Gray warns that wider society is at risk of being infected by this meticulously engineered new language. Nguyen herself admitted this on social network X: Happens to him: “In fact, I recently found myself using 'delve' in my language. Probably because he spends so much time talking to GPT. ” April 8th, X’s official ChatGPT account There's a chime: “I just like digging into what to say.”
Apply our weekly newsletter Get more news coverage in English from EL PAÍS USA Edition
