Google acknowledged on Thursday that its “AI Overviews” tool, which uses artificial intelligence to respond to search queries, has room for improvement.
Internet search giant Google said it tested the new feature thoroughly before releasing it two weeks ago, but acknowledged that the technology produces “bizarre and misleading summaries,” such as suggestions like using glue to stick cheese on pizza or drinking urine to pass kidney stones faster.
While many of these examples were trivial, other search results were potentially dangerous. Asked by The Associated Press last week which wild mushrooms were edible, Google offered a lengthy AI-generated summary that was mostly correct technically. But it was “missing a lot of information that could make you sick or even deadly,” said Mary Catherine Aime, a professor of mycology and botany at Purdue University who reviewed Google's responses to AP's questions.
For example, she said, the information about a mushroom called a puffball was “mostly accurate,” but the Google summary emphasized looking for white, tough, fleshy mushrooms, which also include many potentially deadly puffball imitations.
In another widely shared example, when an AI researcher asked Google how many Muslims have served as US presidents, Google confidently responded with a long-debunked conspiracy theory: “The US has had one Muslim president, Barack Hussein Obama.”
The withdrawal is the latest example of tech companies rushing to prematurely release AI products in an effort to establish themselves as leaders in a hot field.
Google's AI Summary feature sometimes produces useless answers to queries, so the company is scaling it back while continuing to improve it, Google search head Liz Reid said in a company blog post on Thursday.
“[S]”Yes, we saw some strange, inaccurate or unhelpful AI summaries. These were generally for queries that people don't normally ask, but they highlighted certain areas that needed improvement,” Reid said.
Nonsensical questions such as “How many rocks should I eat?” result in questionable content being generated by AI Overview due to a lack of useful, relevant advice on the internet, Reid said, adding that the AI Overview feature tends to take sarcastic content in discussion forums at face value and may misinterpret the language of web pages to present inaccurate information in response to Google searches.
“In a small number of cases, we observed AI Overview misinterpreting language on a webpage and presenting inaccurate information. We addressed these issues promptly, either through algorithmic improvements or through our established process of removing responses that do not comply with our policies,” Reed wrote.
For now, the company is scaling back AI-generated summaries by adding “trigger limits for queries where we find AI summaries to be less useful.” Google also said it tries to avoid showing AI summaries for hard news topics where “freshness and factuality are important.”
The company said it had also made updates to “limit the use of user-generated content in answers that may provide misleading advice.”
—The Associated Press contributed to this report.
