SAN FRANCISCO – Recently, researchers asked two versions of OpenAI’s ChatGPT artificial intelligence chatbot where Massachusetts Institute of Technology professor Thomas Lozano Perez was born.
One bot said Spain and the other said Cuba. When the system instructed the bot to discuss the answer, the bot replied that Spain immediately apologized and agreed with Cuba for the correct answer.
The discovery is the latest potential breakthrough in helping chatbots reach the right answer in a paper published last week by a team of MIT researchers. Researchers proposed using different chatbots to generate multiple answers to the same question and have them debate each other until one answer wins. The researchers found that using this ‘society of mind’ approach brought them closer to reality.
“The language model is trained to predict the next word,” said Yilun Du, an MIT researcher and former OpenAI researcher and co-author of the paper. “They aren’t trained to tell people they don’t know what they’re doing.” Invent an answer instead of admitting it.
The researchers’ creative approach is just the latest attempt to solve one of the most pressing concerns in the exploding field of AI. Despite the incredible leap in functionality that “generative” chatbots such as OpenAI’s ChatGPT, Microsoft’s Bing, and Google’s Bard have demonstrated over the past six months, they still have serious fatal flaws. there is.
Finding ways to prevent or correct what the field calls “hallucinations” has become an obsession among many tech workers, researchers, and AI skeptics alike. The issue has been mentioned in dozens of academic papers posted to the online database Arxiv, and big tech CEOs like Google’s Sundar Pichai have repeatedly addressed the issue. As this technology spreads to millions of people and is incorporated into important areas such as medicine and law, understanding hallucinations and finding ways to mitigate them becomes even more important.
Most researchers agree that the problem is inherent in the “large language models” that drive bots, due to the way bots are designed. They make predictions about what’s most appropriate to say based on vast amounts of data they’ve gleaned from the Internet, but they have no way of understanding what’s actually true.
Still, researchers and companies are working hard on the issue. Some companies use human trainers to rewrite bot answers and feed them back to machines, with the goal of making bots smarter. Google and Microsoft have started using bots to provide answers directly on their own search engines, but they still recheck bots on regular search results. And academics around the world have proposed a myriad of clever ways to reduce the error rate, such as MIT’s proposal to have multiple bots discuss each other.
There is a reason why efforts to ameliorate the problem of hallucinations are urgently needed.
Already when Microsoft launched the Bing chatbot, it quickly began making false accusations against some users, including telling a German university student that it was a safety hazard. Bott adopted his alter-ego and began calling himself “Sydney”. It was basically tracing a student’s question, drawing on all his sci-fi he digested from the Internet about out-of-control robots.
Microsoft ultimately had to limit the number of times a bot could interact with a human to prevent further incidents.
In Australia, ChatGPT was actually a whistleblower in a bribery case, but government officials threatened to sue OpenAI after they announced they had been found guilty of bribery. According to The New York Times, a lawyer last week admitted to using ChatGPT to prepare legal briefs after his arrest because the case the bot so confidently cited simply didn’t exist.
Even Google and Microsoft, which are betting their future on AI and are racing to embed the technology into their wide range of products, have missed hallucinations from their bots during important announcements and demos.
None of them will stop companies from rushing headlong into this space. Billions of dollars are being invested in developing smarter, faster chatbots, and companies are starting to market chatbots as replacements and supplements for human employees. Earlier this month, OpenAI CEO Sam Altman testified before Congress that AI would “cause significant harm to the world” by spreading disinformation and emotionally manipulating humans. said it was possible. Some companies have already said they want to replace their workforce with AI, and the technology poses serious challenges for cybersecurity as well.
AI-powered transcription services have also recorded hallucinations, adding words not spoken in real life to the recordings. Microsoft and Google’s use of bots to directly respond to search queries, rather than sending traffic to blogs and news articles, is a sign of online publishers and content striving to create authoritative information for the internet. It can erode the creator’s business model.
“No one in this field has solved the hallucination problem yet. All models have this problem,” Pichai said in an April interview with CBS. Whether it is possible to resolve it is “a matter of intense debate,” he said.
Hallucinations are, depending on how you look at them, both a feature and a bug in large language models. Hallucinations are part of the functionality that allows bots to be creative and come up with never-before-seen stories. At the same time, they bolster the argument that chatbots are as intelligent as humans by exposing severe limitations of technology and suggesting that chatbots have no internalized understanding of the world around them. weaken it.
“Nothing tells the model that whatever it says should actually be true in the world,” said Yis Kamal, a senior research scientist at Microsoft. The model itself is also trained on a fixed amount of data, so whatever happens after the training is over doesn’t factor into its knowledge of the world, Kammer said.
Hallucinations are nothing new. These have been inherent problems with large language models since they began a few years ago, but other problems, such as AI generating nonsensical or repetitive answers, are also seen as bigger problems. It was done. However, once they are mostly solved, hallucinations are now a key focus of the AI community.
Potsawee Manakul was poking around on ChatGPT and asked for a quick fact about tennis player Roger Federer. It was a simple request that a human could have looked up on Google or Wikipedia in seconds, but the bot kept giving inconsistent answers.
“Sometimes I’ve won Wimbledon five times, sometimes I’ve won eight,” said Manakul, a Cambridge AI researcher and tennis aficionado, in an interview. (The correct answer is 8.)
Manakul and a group of other Cambridge researchers published a paper in March suggesting a system called SelfCheckGPT that asks the same bot multiple questions and compares different answers. If the answers are consistent, the facts are considered correct, but if the answers differ, it may be flagged as possibly containing fabricated information.
When humans are asked to write poetry, we know that it doesn’t always matter that the facts are correct. But when you ask for biographical details about a real person, you automatically know that the answer must be grounded in reality. The chatbot is simply predicting which word or idea will come next in the text string, so it still cannot understand the context of the question.
“There is no concept of whether we should be more creative or less creative,” Manakul said. The researchers showed that they could use their method to filter out factually incorrect responses and rank responses based on fact.
Manakul said it will likely require entirely new AI learning methods that have not yet been invented. Only by building a system based on a language model can the problem really be mitigated.
“It mixes information from many things, so it will produce something plausible,” he said. “But whether that is the case, that is the question.”
That’s essentially what the big companies are already doing. When Google uses chatbot technology to generate search results, it also runs regular searches in parallel and compares the bot’s answers with traditional search results for matches. Otherwise, the AI’s answer will not be displayed either. The company tweaked the bot to be less creative. That means they are not very good at writing poetry or having interesting conversations, but they are less likely to lie.
Google spokeswoman Jennifer Rodstrom said by limiting the search bot to corroborating existing search results, it was able to reduce hallucinations and inaccuracies. An OpenAI spokesperson said a paper produced by the company shows how its latest model, GPT4, has reduced hallucinations over previous versions.
Companies also spend time and money improving their models by testing them with real people. A technique called reinforcement learning with human feedback, in which human testers manually improve the bot’s answers and feed them back to the system for improvement, makes ChatGPT far superior to previous chatbots. It is widely recognized that A common approach is to connect chatbots to databases of factual or more authoritative information, such as Wikipedia, Google Search, or custom collections of academic papers and business documents.
Some leading AI researchers argue that hallucinations should be accepted. After all, we know that humans have bad memories, and we fill in the gaps in our memories without realizing it.
“We can improve it, but we can never get rid of it,” said Jeffrey Hinton, whose decades of research helped lay the foundation for the current swarm of AI chatbots, of the hallucinogenic problem. Until recently, he worked at Google, when he retired to speak publicly about his concerns that technology is moving beyond human control. “We will always be, and they will always be.
–Gerrit de Vink, Washington Post
