Aaron Levy told Business Insider that AI agents have a catch.
The CEO of Cloud Storage Giant Box calls this issue a “context lot.”
The more data you give to your AI model, the more “it doesn't necessarily lead to better results,” Levy said Tuesday. “This model is very confusing and potentially focuses on the wrong parts of the information.”
When a task is dragged, the model could track what it should focus on, leading to worse outcomes, he added.
The flaw occurs when an agent overloads a “context window” with a huge amount of data. This is part of the process of synthesizing information before the model generates a response.
Instead of trusting one super agent, Levy said the smarter approach is to carve out the piece and assign it to a fleet of specialized subagents.
“You'd want to break apart the agents and the context they have,” he said.
“You have multiple agents and you have a close set of contexts for all your goals and specific parts of your workflow,” he added.
It is a counter trend to dreams of Silicon Valley. Levy, who co-founded Box in 2005, said the subagent model “will definitely be the future of a large-scale agent system.”
The CEO also said the key to improving AI performance is to give these models “the most accurate information and the most accurate data.”
Related Stories
Business Insider tells the innovative stories you want to know
Business Insider tells the innovative stories you want to know
“You have to be very accurate in your instructions, and you need to give the model an amazing amount of correct context to work, but in reality too many contexts, it's worse,” he added.
AI agents are far from perfect
Silicon Valley is buzzing about AI agents, and businesses are racing to use it for increasingly sophisticated and multi-step tasks.
Follow-up with prospects and buyers from Regie AI's “automatic pilot sales agent”; Devin, cognitive AI, tackles complex engineering tasks, while Big Four Professional Services Firm PwC has introduced an “agent OS” to allow various agents to coordinate with each other.
But the reality is a hassle. In theory, agents can solve problems, perform tasks, and become smarter as they learn. In fact, the more steps you take, the more vulnerable the process becomes.
Researchers warn that agent errors are common and complex at each step they take.
“Errors at every step can derail the entire task. The more steps you get involved, the more likely you end up with something,” wrote Patronus AI, a startup that helps companies evaluate and optimize AI technology, on their blog.
The startup built a statistical model that found that agents with an error rate of 1% per step can be combined to match the likelihood of 63% of error in the 100th step.
Still, the company said guardrails such as filters, rules and tools to identify and remove inaccurate content will help reduce error rates. Patronas AI said the slightest improvement “can cause error probability.”
