“Human involvement” is a common recommendation for organizations that use artificial intelligence for operations where there is no room for error.
But what happens when it is impossible to keep constant surveillance on humans?
It’s a challenge many business leaders are beginning to grapple with as the use of AI increases within organizations and companies delegate more tasks to agent AI.
“The speed of manufacturing has increased tremendously,” Zach Mayberry, chief technology officer of online sports betting platform DraftKings, said during a panel discussion. Fortune’s This month’s flagship technology conference is Brainstorm Tech.
Mayberry said his company already handles trillions of transactions and highly distributed workloads. When we introduce agent AI, where AI agents communicate directly with other AI agents, the amount and complexity of operations becomes too overwhelming for traditional approaches.
“You can’t put humans in the loop,” Mayberry said. “There will never be enough people to insert themselves into all these loops.”
Mayberry was just one of several business leaders at Brainstorm Tech who discussed the challenges of managing AI in mission-critical situations.
“In a high-stakes environment like healthcare, if you’re a retailer, getting the wrong T-shirt size is not an issue; it’s life on the other side,” said Lashonda Anderson-Williams, chief customer and sales officer at Salesforce.
While there is no one-size-fits-all solution to these challenges, many of the panelists described techniques and frameworks that have proven successful.
Anderson-Williams said it’s important to take a hard look at AI use cases and understand what the end result is.
Equally important is establishing an appropriate governance framework. That means clear policies and a set of rules that govern where and how AI is allowed to operate, how it is designed, and who is responsible for different parts of the process. As companies scale the use of AI and agents from small-scale experiments to widespread deployments with high risk, modern governance frameworks are essential.
“A lot of people were running and buying different tools and technologies and just bolting them on. There was no governance on how the technology was being applied,” Anderson-Williams said.
DraftKing’s Maybury said having a strong AI governance foundation in place provides important safeguards and helps reduce risk. That might mean overhauling existing processes and making changes, modifications, or enhancements to old governance rules.
“It has to be scalable governance,” he said.
Anthony Moisant, Indeed’s chief information security officer, echoed Mayberry’s comments about the challenge of keeping humans informed in a massive job search service used by 645 million job seekers and 3.5 million employers. He suggests continuously testing processes involving AI to assess how the results compare to desired outcomes.
It’s also important to consider the types of situations in which AI will be deployed, says Diya Jolly, chief product and technology officer at accounting software company Xero. Does it require judgment or is there a clear answer?
“If the results are conclusive, you can probably leave the agent pretty far,” Jolly said, noting that the results in such situations can be easily tested and measured against the desired results. But, she said, “When there’s a sense of self-judgment in the decision, that’s when it becomes really difficult to let people out.”
