Despite the enthusiasm of some CEOs, AI agents are not always up to the task.
One of the causes is hallucinations.
An agent is a software program that can independently perform tasks, such as invoice matching, without the need for a human to do it. That’s wonderful. That’s why companies involve agents in operations across sales, finance, supply chain, and engineering.
While agent systems can efficiently complete individual tasks, they can become vulnerable when scaled up.
Nicolas Darvaux-Garneau, a former Google executive and author of “Be a Sequoia, Not a Bonsai,” says errors can quickly become a bottleneck as hallucinations increase in agent interactions.
If the hallucination rate of a single agent is 5%, it is difficult to daisy chain multiple agents without high risk of error. That’s because the risk increases exponentially, he told Business Insider.
“The only way to significantly increase productivity is to reach a day where we can trust AI to have fewer hallucinations,” Darvaux-Garneau said.
For now, he said, that’s why general-purpose autonomous agent AI is so exciting.
Nevertheless, Darvaux-Garneau predicts that hallucinations will be “mostly resolved” five years from now.
If that happens, he said, the productivity gains some companies are seeing from AI could be much larger, ranging from 10% to 30%, perhaps as much as 10x to 100x.
These profits are important because the greater the profits, the more likely the company will restructure its operations, including possibly requiring fewer employees.
Even if companies do not cut staff, many roles are likely to change, especially desk jobs.
Scaling up requires rethinking corporate structure
The challenge of broadly scaling agents and AI is one reason why we expect companies to still need talent for jobs that can sometimes be seen as risky, like coding.
He says that talented software engineers can increase their productivity by 30% to 40% by leveraging AI. Darvaux-Garneau said while this is a nice increase, it’s not a reason for companies to lay off all their engineers.
He says it’s natural for workers to fear losing their jobs to AI, especially if leaders emphasize improving productivity.
Darvaux-Garneau said the CEO’s main job when it comes to AI is to create the right environment for adoption, including training and making sure employees feel “psychologically safe.”
That’s why, he said, leaders would be wise to reframe the conversation to say, for example, increasing the employee base by 30% while doubling the size of the company, rather than proposing to implement AI and eliminate 20% of the workforce.
Management knows change is coming
One of the challenges for many leaders is understanding what their new roles will look like thanks to AI, Mukshit Ashraf, Accenture’s group chief executive officer of strategy, told Business Insider.
That has proven difficult, at least in part, because some current job descriptions may change and others become unnecessary. Still others will need to be “fundamentally redefined,” he said.
“What companies have generally done is put AI in the hands of the people who are responsible for their existing jobs,” Ashraf said.
That might mean automating some tasks or using AI to quickly learn how to find information and perform competitive analysis.
“But how would that fundamentally change my day?” he said.
Ashraf said companies are struggling if they tend to think about implementing AI within their current ways of working, rather than redesigning their organizations with an eye to the future of work.
“It’s not about layering AI onto existing roles; it’s about rethinking all these processes and activities,” he said.
Ashraf said that despite the challenges of redesigning how companies work, it is encouraging that many executives recognize the need to help employees develop the necessary skills to prepare for the changes that will result from companies’ decisions to prioritize AI.
While AI may still have its limits, its rapid advancement means that employees themselves should prioritize staying up to date. Otherwise, you risk falling behind, Alex Salazar, co-founder and CEO of AI infrastructure startup Arcade, told Business Insider.
A year ago, he said, the coding agents the company’s software engineers now use couldn’t do anything they can do now.
“What does that mean for your job as an engineer?” Salazar said. “We have to evolve.”
Do you have a story to share about your career? This reporter can be contacted at: tparadis@businessinsider.com
