Disaster can occur when you train one chatbot on another
Photo/Getty Images
Multiple whistleblowers have said that people who are paid to provide high-quality conversations and tests to train new AI models are fraudulent and instead use chatbots like ChatGPT to do that work. new scientist. This seemingly widespread practice risks undermining the future of AI, as it can lead to the “breakdown” of more advanced models.
Most of the AI models in operation today were trained based on text and data collected from the internet. But as models scale up and require more training data, AI companies have started using workers to talk to and test the AI, with the hope that the resulting high-quality data will improve the power and usefulness of future large-scale language models (LLMs).
These workers are typically hired by third parties rather than directly by AI companies, and often work for low wages without full-time contracts. According to an employee named Alice*, this can motivate them to take shortcuts, such as using chatbots to complete tasks faster, even though it’s against company policy.
“It’s very prevalent. I think they care because every company I’ve ever worked for has clear guidelines about it and is clearly trying to catch people. But I don’t think they can stop it,” Alice says.
Alice says He says he doesn’t feel guilty “at all” about using ChatGPT to complete training tasks, and that it’s easy to work around the issue as long as you tell the chatbot to avoid the usual obvious signs of AI output, such as heavy use of em-dashes. “Only the sloppiest users get caught,” she says. “If you have some awareness of the characteristics of an AI, you can tell it not to use it in your output. At that point, what do you do?”
“If these companies want quality data, they should offer quality contracts,” Alice says. “Instead, they hire people who are struggling, keep them on for as long as possible, and then dump them when the project ends without warning.”
Another employee, Bob*, worked for a training platform called Outlier. Initially, he was tasked with training the AI, which he claimed to have been fraudulently using for that purpose, and was later promoted to a leadership role, where part of his job was to catch others doing the same thing.
“Management vacillated between mild tolerance and outright banning,” Bob says. Outlier’s employees are tracked with a tool called Hubstaff, which takes screenshots of their desktops at random intervals to ensure they are indeed performing tasks as ordered. Bob looks for evidence of an AI model in those screenshots.
“People will have it [AI models like ChatGPT] “Obviously it’s on the taskbar because it’s open in another tab or minimized,” Bob says. [AI use] Away. “
Outlier, which is owned by Scale AI, did not respond to a request for comment. Scale AI claims on its website that it does work for tech giants like Meta and Cisco, but neither responded. new scientistThis is a comment request from . Bob said he was personally working on the project for Google, but Google also did not respond to a request for comment.
Carol*, another employee who has worked on multiple platforms, says her use of AI started by checking for things that violated long guidelines for tasks. Violations can result in expulsion from the project and loss of income.
“I was scared of not having a source of income, but then it became easier to do everything through the LLM,” Carol says. “A lot of the projects I do now are creating scenarios, so I use one LLM to create the scenarios, and then use another LLM to create the files that follow the scenarios. I feel guilty, but like I said, it was more about making sure I didn’t make mistakes in the beginning.”
“I’m worried about whether it’s really successful. [AI] bad. I thought training myself using a model negates some of the value,” says Carroll.
Mark Lee from the University of Birmingham in the UK says research shows that AI models “break down” when trained recursively on AI-generated content. When this happens, the model’s power decreases dramatically, rendering it useless. This process is sometimes known as AI cannibalism or AI inbreeding.
“That’s kind of the worst-case scenario, and that’s probably not what’s happening in the real world,” Lee says. “Humans are still a minority, and having 10% or so of human data will alleviate that and avoid model collapse.”
However, Lee said the misconduct carried out by these employees is not without negative consequences and will affect business results. “Rather than it being a catastrophic situation, it shows that AI is not very good at performing human-like tasks. This is a problem because I don’t think the models are as good as they could be.”
*Names have been changed to protect personal information
topic:
