OpenAI announced Wednesday that it will play a role in combating “hallucinations” in AI with a new way to train AI models.
The research comes at a time when misinformation caused by AI systems is more hotly debated than ever in the run-up to the generative AI boom and the 2024 US presidential election. Last year, OpenAI accelerated the generative AI boom by releasing ChatGPT, a chatbot powered by GPT-3 and GPT-4, becoming the fastest growing with over 100 million monthly users in two months. It reportedly set an app record. to date, microsoft has invested over $13 billion in OpenAI, valuing the startup at around $29 billion.
AI’s hallucinations are similar to OpenAI’s ChatGPT and GoogleThe bard has completely fabricated information and behaves as if he is spouting facts. One example: In his Bard promotional video that Google released in February, the chatbot makes false claims about James’ Webb Space Telescope. Most recently, ChatGPT cited a “fake” case in a filing in New York federal court, in which the New York attorneys involved could face sanctions.
“Even state-of-the-art models tend to be false, and to fabricate facts in moments of uncertainty,” the OpenAI researchers wrote in their report. “These hallucinations are particularly problematic in areas that require multi-stage reasoning, as a single logic error is enough to derail larger-scale solutions.”
Potential new strategies for OpenAI to combat hoaxes: training AI models to reward each correct step of reasoning in reaching an answer, rather than rewarding the final correct conclusion. to The researchers say the approach is called “process monitoring” rather than “outcome monitoring,” and this strategy leads to more explainable AI because it forces models to follow a more human-like “thought” chain approach. It is said that there is a possibility.
“Detecting and mitigating logic errors and hallucinations in models is a key step in building a tuned AGI. [or artificial general intelligence]Karl Kobbe, a mathgen researcher at OpenAI, told CNBC. OpenAI said it didn’t invent the process monitoring approach, but it helped drive it forward. “The motivation behind this study is to address hallucinations in order to make models better capable of solving difficult inference problems.” ”
Cobb said OpenAI has released an accompanying dataset of 800,000 human labels that it used to train the model mentioned in the research paper.
Ben Winters, a senior adviser to the Electronic Privacy Information Center and leader of the AI and Human Rights Project, expressed skepticism and told CNBC that he would be interested in seeing the full dataset and accompanying examples.
“I doubt that this alone would significantly alleviate concerns about misinformation and misleading results in real-world use,” Winters said. “It’s definitely important whether they have plans to implement what they’ve found here,” he added. [into their products]If not, it raises some pretty serious questions about what they’re trying to expose to the public. ”
It’s not clear whether the OpenAI paper was peer-reviewed, or in some other form, so Suresh Venkatasbramanian, director of Brown University’s Center for Technology Responsibility, told CNBC that the study was preliminary above all. He said he thought it was an important observation.
“This issue needs to be resolved within the research community to say anything with certainty about it,” Venkatasbramanian said. “In this world there are a lot of results that come up very regularly, but because of the general instability of how large language models work, in certain settings, models and contexts, What might work may not work in another setting, model, or context.”
Venkatasbramanian added, “Some of the hallucinations that people are concerned about are [models] Create citations and references. There is no evidence in this paper that this works…I’m not saying it won’t work. This paper does not provide that evidence. ”
OpenAI did not respond to a request for comment asking whether the research has been reviewed by an external party in some way or when, if any, the company plans to implement new strategies in ChatGPT or other products.
“It is certainly welcome to see companies trying to tinker with the development of their systems to reduce these types of errors. I think it’s about interpreting it as research,” Sarah Myers-West, managing director of the AI Now Institute, told CNBC.
West further added,[OpenAI is] Although this paper publishes a small dataset of human-level feedback, it does not provide basic details about the data used to train and test GPT-4. So, despite the direct impact these systems already have on people, there is still a great deal of uncertainty that hinders meaningful accountability efforts in the field of AI. ”
