From hallucinations to rogue agents, there are some obvious risks to using AI.
Still, most companies can’t afford to ignore the AI revolution. Dealing with this troubling reality is a fundamental challenge for today’s business leaders, and executives from some of the largest companies gathered at Fortune Brainstorm Tech in Apsen, Colorado, to share their insights and experiences.
At the top of the priority list is accountability. This means being able to track every step an AI or agent AI system takes in performing a particular task, and re-trace it if necessary.
“The key thing we’re concerned about is how can we build a system that is as accurate as possible,” said Edwin Olson, founder and CEO of self-driving technology company May Mobility. “But importantly, we know that we’re going to make mistakes eventually, so how can we be more transparent and self-reflective so that we can talk to regulators about understanding why we made mistakes and how we’ll know that we’ve fixed that problem going forward.”
Caitlin Halferty, chief data officer at Thomson Reuters, echoed similar sentiments, emphasizing the importance of transparent output from AI. “I do this with my team and myself, and I encourage my clients to do this as well, making sure there is a way to be able to validate the output of the models they are using.”
With a portfolio of AI-enabled services for professionals in areas such as legal and tax compliance, Thomson Reuters needed to focus on AI accountability early on. Halferty said transparency is one of the four main pillars of what the company calls “fiduciary grade” products, along with data privacy and security, subject matter experts, and trusted content.
Another important technique cited by several panelists is to design systems that can effectively control each other. At May Mobility, Olson said this includes equipping self-driving cars with systems that can simultaneously simulate and evaluate different scenarios and select the best option.
However, such systems can also be used in corporate environments and daily workflows. Elena Kovochko, founder and CEO of Trustguard AI, calls this the “LLM as judge” approach and uses a newsroom analogy to explain how it works.
“There’s one person or agent who does the writer’s job, and then there’s another person or agent whose job is the editor, whose sole purpose is to find mistakes and inaccuracies that the writer may have missed. Essentially, we want to design the LLM system this way too, so that the system can self-improve.”
But the key, Kvochko adds, is that validation needs to be comprised of a separate AI system. “You don’t want an AI to grade your work,” she said.
Having smart structures for AI validation will become increasingly important as technology performs more tasks and surpasses the human ability to validate all work.
“You end up in this space where you have so much work done and so much work to audit that you can’t really be accountable,” said Gregor Stewart, chief AI officer at SentinelOne.
He pointed to computer coding, which he said is about a year ahead of other industries. Rather than having humans verify 10,000 lines of code created by AI, the team is figuring out how to have agents emulate some of the processes developed decades ago for humans in safety-critical industries.
“I think a lot of the technologies that we developed for safety-critical technologies will be brought back into the very average practice,” Stewart said.
