AI competition can sometimes feel like a numbers game.
Earlier this month, McKinsey & Company CEO Bob Sternfels made a surprising announcement. He said the company has more than 60,000 employees, including 40,000 human employees and 25,000 AI agents.
For Sternfels, this was an example of McKinsey’s all-in approach to AI. other people in the industry, but, The surprising numbers actually say little about the success of the company’s artificial intelligence implementation.
Dan Priest, chief AI officer at PwC, told Business Insider that measuring a company’s use of AI by the number of agents it has is not the best metric.
“I developed this kind of bragging about how many agents I have, or how many agents I actually run,” he said. “I think that’s probably the wrong move.”
The value of AI deployment is measured by the quality of agents, not the quantity, he said.
Priest said one way to do that would be to look at how many agents have authority for certain tasks, which would encourage humans to use them. The other is to assess the number of humans using these agents to perform tasks to achieve high-priority outcomes for the company. An example is improving the customer experience by transforming call centers.
Over the past two years, agents have dominated the way companies talk about AI adoption. Priest said the right approach is to focus on agents. “Agents are now figuring out how best to extract value from AI,” he said.
But humans still power the workforce, and a better way to measure the value of agents is by how effectively people utilize them, not just the amount of work they potentially automate.
At PwC, approximately 82% of employees actively used the company’s AI tools. Priest said PwC has embedded AI agents across its teams, and the company tracks how agents interact, how accurately they complete tasks, and whether processes are faster, higher quality, or better performing. The human is responsible for reviewing the agent’s output and providing feedback.
“Humans still have a responsibility,” he said. “It’s human to certify. It’s human to license. It’s human to empower.”
Priest said PwC and its clients initially took a bottom-up approach to AI implementation. He said many business leaders are trying to “crowdsource” approaches to hiring from their employees because they don’t have the answers themselves.
The result was a “pretty disappointing” return on investment, he said.
Priest said the move to a “top-down” approach has become more effective, allowing the company to focus on a smaller set of tasks and fewer agents with deeper mastery.
“That agent, I gave them access to a certain data set,” he said. “I gave them permission to perform certain tasks. I gave them permission to produce certain results. Those permissions are monitored, expired, and managed.”
