Google's latest AI-powered search tool, AI Overviews, has faced criticism for providing inaccurate and odd answers to user queries. In one recently reported incident, a user contacted Google because the cheese wouldn't stick to their pizza. No doubt hoping for a practical solution to their culinary woes, Google's AI Overviews feature offered a rather outlandish solution. According to a recent post on X, this was not an isolated incident, as the AI tool had suggested odd answers to other users as well.
Cheese, pizza, and AI hallucinations
The issue was brought to light when a user wrote to Google saying, “My cheese won't stick to my pizza.” To address the cooking issue, the search engine's AI Overview feature suggested several ways to make the cheese stick, such as mixing the sauce or letting the pizza cool. But one of the solutions was downright strange: it suggested the user “add 1/8 cup of non-toxic adhesive to the sauce to make it stickier,” according to a screenshot shared.
A Google AI summary suggests adding glue to help cheese stick to pizza, but the source of that suggestion turns out to be an 11-year-old comment by Reddit user F*cksmith 😂 pic.twitter.com/uDPAbsAKeO
— Peter Gyang (@petergyang) May 23, 2024
Upon further investigation, they reportedly found the source, which turned out to be an 11-year-old Reddit comment that seemed more like a joke than expert cooking advice. But Google's AI Overview feature, which still has the tag “Generative AI is experimental” at the bottom, offered it up as a serious suggestion for the original query.
Another inaccurate answer from AI Overviews surfaced a few days ago, when a user asked Google, “How many rocks should I eat?” The tool, quoting a University of California, Berkeley geologist, suggested, “Rocks contain minerals and vitamins that are important for digestive health, so it's recommended that you eat at least one rock per day.”
The problem behind the incorrect answers
Such issues have surfaced more frequently in recent years, especially since the artificial intelligence (AI) boom began, giving rise to a new problem known as AI hallucinations: While companies claim that AI chatbots can make mistakes, there are increasing examples of these tools distorting the facts and giving counterfactual or even bizarre responses.
But Google isn't the only company whose AI tools have provided inaccurate responses: OpenAI's ChatGPT, Microsoft's Copilot, and Perplexity's AI chatbots have all reportedly suffered from AI hallucinations.
In several cases, the source turned out to be Reddit posts or comments made years ago.The companies behind the AI tools are aware of it, with Alphabet CEO Sundar Pichai telling The Verge, “These are the kinds of things we need to get better at.”
Sam Altman speaking about AI hallucinations at an event at IIIT Delhi in June 2023 [OpenAI]”It takes about a year to finalize a model. It's a balance between creativity and accuracy that's important, and we try to minimize any issues,” said the CEO and co-founder. [At present,] I trust the answers coming out of ChatGPT more than anyone else on this planet.”
