Google revamps AI search overview after strange results

The Alphabet division pulled the plug on the new feature after users reported strange and incorrect answers, such as recommending eating rocks for health benefits or using glue to stick cheese to pizza.

The AI ​​Summary feature, introduced earlier this month, provides users with a question-answering statement at the top of Google search results, before the list of links. Google said it was able to observe patterns where issues arose and has made 12 technical improvements to the feature.

“Like our users, we hold ourselves to high standards, so we expect, appreciate and take your feedback seriously,” Liz Reid, Google's vice president of search, said in a blog post published on Thursday.

It's the second time this year that Google has had to publicly course-correct after a major AI release. CEO Sundar Pichai said in February that the company was making structural changes after Gemini, a chatbot based on Google's cutting-edge AI technology, angered users by generating ahistorical images and blocking requests for depictions of white people. The controversy bled into a broader backlash over the chatbot's answers to a range of philosophical questions.

Google is embracing generative AI — a type of software that can create text and other media — to fight growing competition for its core business. The rollout of AI summaries is seen as a key step in protecting the search market, where the company makes hundreds of billions of dollars from advertising.

The search giant said it tested the AI-generated summaries thoroughly before releasing them, “but nothing beats millions of people using the feature for many new searches,” Reid said in a blog post. “We've also seen some nonsensical new searches that appear to be designed to produce false results.”

Google also said there were a number of fake screenshots purporting to show overview results from its AI, and urged anyone who comes across screenshots showing dangerous results to search and check for themselves.

In response to strange, inaccurate, or even unhelpful summaries, Google built better detection systems to identify meaningless queries that don't show AI-generated results, and also limited user-generated content that could provide misinformation.

Reid said user feedback shows people are more satisfied with the results when they see AI summaries. He said Google will continue to improve the feature and strengthen its protections. “We're very grateful for your ongoing feedback,” he wrote.

Write to Aaron Tilley at aaron.tilley@wsj.com