According to at least 14 academics, AI model collapse – the expected decline in quality from machine learning models that recursively train on their own outputs – is inevitable.
The risk that ongoing generative AI output, known as synthetic data, dilutes the organic data created by humans and undermines the performance of models trained on this increasingly fabricated corpus was raised last year by another group. It was highlighted in a paper titled: Recursion: The model is trained on generated data and then forgotten. ”
Ilya Shumailov, lead author of the paper, said: register This phenomenon was also reported in other studies earlier this year.
Now another set of Excellencies, Matthias Gerstglasser, Rylan Schaefer, Apratim Dei, Raphael Rafailov, Henry Slight, John Hughes, Tomasz Korbach, Rajashree Agrawal, Dhruv Pai, Andrei Gromov, Daniel Roberts, Diyi Yang, David Donohoe and Sanmi Koyejo will be in contention. Given the way model training actually happens, the question of training an AI on data created by the AI is irrelevant.
This latest batch of bakers' 12 plus one, with participants from Stanford University, AI safety group Constellation, the University of Maryland, College Park, MIT, and Sequoia Capital, is the author of “Is Model Breakdown Inevitable? In a paper titled “Breaking the Curse,” he argues that there is no need to worry. Recursion by accumulating real and synthetic data. ”
Although the authors claim that their research results do not necessarily reflect the positions or policies of their funders, some of these organizations are supported by grants from commercial organizations such as OpenAI and Google. It is noteworthy that the company acknowledges the support provided by
Gerstglasser, a postdoctoral fellow at Harvard SEAS and a visiting postdoctoral fellow at Stanford University, outlined the arguments he and his colleagues hope to make on social media.
“As AI-generated content becomes more prevalent on the internet, there are growing concerns that future AI models will be trained on this 'tainted' data,” he argued. “This is like a virus that can infect the entire AI ecosystem.
“Many experts have warned that this could lead to an AI doomsday scenario. If models continue to get worse and worse with each generation, we could face an 'AI apocalypse.' Yes! But don’t panic yet…”
Gerstglasser said previous research has warned about this “doomsday scenario,” but all of that research assumes that subsequent generations of AI will be trained solely on synthetic data generated by previous generations of models. claimed to be based on.
He argues that legacy data is not simply discarded. They are more likely to accumulate rather than be replaced with each generation. The synthetic data is simply mixed with the organic data, and the resulting model continues to run.
“Our findings extend these previous studies by showing that as data accumulates and models are trained on a mixture of 'real' and synthetic data, model collapse no longer occurs. ” said Gerstglasser. other He declares in “Is model collapse inevitable?” paper.
”[T]These results suggest that the “curse of recursion'' may not be as dire as previously portrayed if synthetic data is accumulated along with real data, rather than replacing real data with synthetic data alone. It strongly suggests that there is. ”
But Elvis Dohmatob, Yunzhen Feng, and Julia Kempe, authors of a companion paper titled “Demystifying Model Collapse: The Case for Regression,” say synthetic data can be added to model training without producing consequences. I don't agree with the opinion.
It's all about scale
said Julia Kempe, professor of computer science, mathematics, and data science at New York University's Data Science Center and Courant Institute for Mathematical Sciences. register “Is model collapse inevitable?” Note that this paper is misguided in its conclusions, which rely primarily on research conducted by her and her colleagues.
“Typically, when you train a model on a large amount of data, the more data you train on, the better and better the model becomes,” Kempe explained. “This relationship is called the 'scaling law' and has been shown to hold true empirically in many settings and theoretically in some models.
“In our paper, we refer to the number of times (we will call the number n ) shows that its performance does not follow the usual scaling laws; rather, it effectively behaves as if it were trained on only n parts of the original data.
“For example, if you repeat training and synthesis 10 times and then train using the data from the last model, you will only get the performance you would get if you trained 1/10th of the time.th The original data is much worse!”
Yunzhen Feng, a data science doctoral student at New York University and one of Kempe's co-authors, also disagreed: “Is model collapse inevitable?” His paper and its suggestion that model collapse is negligible.
If your goal is to maintain good performance, it may be desirable to consistently use the original dataset.
“If the goal is to maintain good performance, it may be desirable to consistently use the original dataset that has already been saved and selected before introducing synthetic data,” Feng explains. Did.
“Our goal is to maintain the benefits of scaling,” Feng continued. “In our scaling method, increasing the dataset size by a factor of 10 using clean data results in better scaling. Conversely, using synthetic data not only loses these benefits, but also reduces performance. Therefore, we do not agree with these data.
Feng also pointed to another paper, “Tale of Tails: Model Collapse as a Change of Scaling Laws,” by Dohmatob, Feng, Pu Yang, Francois Charton, and Kempe. register: “From a scaling perspective, we argue that model collapse on AI data is two-fold: the performance benefits typically gained by adding human data are lost; and recursive degradation and retraining on AI data over generations.”
Feng points out that while there are various strategies that can be implemented to prevent recursive degradation, there is a performance impact. “I don't think that's enough to claim that we can avoid collapse.”
counterpoint
Shumailov and his colleagues Zakhar Shumailov, Yiren Zhao, Yalin Gal, Nicholas Papernot, and the late Ross Anderson, in their “Curse of Recursion” paper, put forward the idea that AI is doomed to devour itself. I should say that I wasn't actually promoting it. Their conclusion is more nuanced: model breakdowns can be mitigated by investing money to ensure data quality, which will be easier to spot in large companies than in small businesses. That's what it was.
Asked about Gerstglasser's findings other“In principle, it doesn't really negate what we've shown,” Shmailov said. “In a simple model, we show that some effects can be weakened. This does not come at a cost. Note that it continues to grow and does not solve any problems for ordinary users, who will not have the ability to store their data long-term.
AI breakdown is inevitable, but so is model performance. ®
