- At the Google IO conference, CEO Sundar Pichai teased where AI will go next.
- It will all be about agents who are good at reasoning and can act on our behalf.
- What would you do if Google searched for you?
What if Google Assistant was actually an assistant?
That's a question the company finally began to answer at the 2024 Google IO conference on Tuesday, where the search giant made dozens of AI announcements.
As Google and other AI companies advance the way these systems ingest images, video, audio, and text, these systems can do more of the hard work from smart chatbots. We're starting to see how it's evolving into a sophisticated tool.
That's something Google CEO Sundar Pichai is thinking about a lot right now.
“I think of AI agents as intelligent systems that exhibit reasoning planning and memory,” Pichai said in a roundtable with reporters ahead of this week's developer show.
“They can think ahead and collaborate across software and systems to get things done on your behalf and, most importantly, under your supervision.”
In other words, AI agents are best suited to transform this technology from “nice-to-have” to “must-have.”
project astra
Pichai said Google is in the “very early stages” of developing this, but promised that IO will give a glimpse into “the direction of agents” across Google's products. There have been a few, but the big one that people talk about is the Astra program.
Astra is what Google Assistant was meant to be from the beginning. You can also think of it as a smarter version of Google Lens. It uses real-time computer vision capabilities to allow you to ask questions about what you can see and hear around you.
“We've always wanted to build universal agents that help people in everyday life,” said Demis Hassabis, head of Google DeepMind. “Imagining an agent who sees and hears what we do best, understands the situations we are in, responds quickly in conversations, and makes the pace and quality of interactions feel more natural. please look.”
Google showed a demo in which someone turned on the camera, held up their phone, and asked the AI voice assistant what it saw. For example, they pointed it out the window and asked, “What region do you think I'm in?” It was actually the location of Google's Kings Cross office in London.
Hassabis stressed that the demonstration video was recorded in “real time.” Google received a lot of backlash in December when a Gemini AI model demo was found to have been edited, so Google needs to emphasize that it can actually do this, especially after OpenAI showed off a similar demo on Monday. was there.
Without any real manipulation here, the Astra would certainly be impressive and probably a big takeaway from the show. But there are other ways these AI agents could emerge in the near future.
Google has teased a combination of updates that will soon improve the functionality and proactiveness of its Gemini AI chatbot. Some of this is being freed up as Google continues to increase its context window (the amount of information that large language models can take in at once).
Let's say you want to know something that's buried deep in a very long set of documents and you don't want to spend hours digging through it. A large context window lets you share all your documents with Gemini before asking questions. The model can respond quickly based on all the information it has ingested.
tentacles and tailoring
But where Google really has an edge in delivering agent-like quality is in traditional products.
Already with its tentacles in many aspects of our lives, from email to search to maps, Google is integrating all of its knowledge about you and the world around you to not only answer your queries, but also respond to them. can be customized.
“My Gemini is actually supposed to be different from your Gemini,” said Sissy Xiao, head of Gemini and Google Assistant.
Later this year, Geminis will be able to plan their vacations at a more detailed level, Xiao said. The idea is that you can enter all your specific desires (like hiking, hate heat, allergy to shellfish, etc.) and Gemini will send you back a detailed itinerary. Chatbots can already do this kind of thing, but if given permission, Gemini will have access to flight information, Gmail travel confirmations, and possibly hotels, which it can use to answer questions. can be notified.
“The AI assistant should be able to solve complex problems and perform actions on behalf of the user, and it should also feel very natural and fluid when using the AI assistant,” Xiao said. states.
A lot of what agents do is remove steps and shorten tasks. Google has this in mind when it comes to search, too, evolving its most valuable product to spit out answers created using generative AI.
The company is rolling out a custom version of Gemini built specifically for search, combining web knowledge with the multimodal capabilities of AI models and a huge context window.
Liz Reid, Google's head of search, gave an example of Google Lens being used to capture video of a record player. The user asks Gemini why the arm won't stay in place. Google will return exact instructions for that particular turntable.
As always, Google has some confusing branding to iron out, including Lens, Assistant, Gemini, and Astra. But eventually, many of these things will eventually come together. On Tuesday, Google gave us a hint at what this logical conclusion might be.
As Reid says, “This is how Google searches for you.”
