A year may seem like a long time in enterprise technology, but the past 12 months have completely rewritten enterprise architecture strategies in the field of artificial intelligence (AI).
Speaking to Computer Weekly on the sidelines of the Dell Technologies World conference in Las Vegas, John Rhodes, Dell’s global chief technology officer, said the maturation of agent AI is forcing IT leaders to rethink infrastructure, data management, and operational costs.
“We’ve changed the premise in that using AI is no longer a one-off task like a chatbot,” Roese says. “It’s about passing a goal to an AI system, and that’s what agents do today,” Rose said.
As an example, he cited Google’s search engine redesign. “You give it a purpose, it does a search, it creates an entire page. These are all agents working to achieve your purpose.”
Because agent AI has a much better user experience and humans are the leaders rather than the doers, companies are scrapping old generative AI use cases and rebuilding them as agent workflows.
Busting the myths of GPU training
The early AI boom led to a rush to secure graphics processing units (GPUs) for model training, but Roese said enterprises’ infrastructure requirements are very different from those of hyperscalers.
“There’s a myth that enterprises need thousands of GPUs,” Roese explains. “Our largest internal Dell workloads are only on 16 GPUs, supporting 40,000 employees. We don’t need thousands of GPUs in our enterprise. Each workload, agent, or project requires only a few GPUs, and sometimes as little as half a GPU.”
That’s because many enterprise AI assets are focused entirely on inference rather than training. “For an agent, all you need is inference. There is no training for the agent.”
That said, the architecture required for inference workloads is also changing. When companies were building chatbots, this architecture made them very CPU-intensive. However, AI agents use components that are not native to the GPU, such as external tools, communication protocols, and knowledge graphs.
“When you move to an agent, it pretty much balances out,” Rose said. “The CPU and GPU counts are very similar, with probably a CPU for every two GPUs. Instead of just stacking GPUs to build an AI infrastructure, you build it using GPUs and traditional CPU compute.”
Frontier model and edge with air gap
Businesses are also benefiting from changes in the way they deploy powerful AI models. A year ago, the most capable frontier models were locked behind cloud APIs (application programming interfaces).
Our largest workload within Dell resides on only 16 GPUs and supports 40,000 employees. Enterprises don’t need thousands of GPUs because each workload, agent, or project requires only a few GPUs, or even half a GPU.
John Rose, Dell Technologies
But now that hyperscalers can run top-level models on-premises through services like Google Distributed Cloud, private models can now be deployed across multiple topologies, Roese said. “You can have it in a virtual private cloud, in a data center, and you can air-gap it with everything else. A year ago, we didn’t have those options, except for APIs,” he said.
At the same time, AI is moving to the edge in a structured way. Roese pointed to the recent emergence of agent frameworks like OpenClaw that run natively on devices and AI PCs. “We finally have a structure for running agents on devices, which is incredibly powerful and not just a fad.”
Rebuild the data layer
Meanwhile, data strategies are evolving in parallel with the development of agent AI. Rose warned that bolting standard data storage systems onto AI computing clusters is no longer sufficient to meet the performance demands of AI agents.
Instead, organizations must build knowledge and context layers consisting of vector databases, graph databases, and data annotation tools. These layers cannot be kept separate and must be deeply integrated into computing.
“One of the performance bottlenecks is that the GPU can’t get the data fast enough to do the work,” Roese said, adding, “The GPU you’re paying for is sitting idle waiting for data.” To reduce this latency, Roese said Dell’s AI data platform is now embedded in Nvidia’s Cuda-X interface to efficiently run data layer services directly at GPU speeds.
Master tokenomics and model routing
IT leaders must also manage the cost of AI consumption, even though the cost per token is expected to decline over time as more model deployment options become available with different pricing mechanisms. Companies need to treat AI workloads as an arbitrage game because “there’s no way it’s going to be cheaper to run AI,” Roese says.
He cited specification-driven development, where AI writes software based on markdown documents, as an example, noting that if an agent framework generates a large number of coding tasks and blindly sends them to a top-level model, it can lead to higher bills for companies.
However, with model routing, companies can ensure that complex planning tasks, such as creating software specifications, are sent to expensive frontier models, while routine coding tasks are sent to smaller, on-premises open source models whose only operating costs are energy.
“When you build software and do spec-driven development, you may need four or five different economic paths to ultimately achieve the highest overall economic efficiency,” Roese said. Mastering model routing can be a competitive differentiator and help reduce product development costs, he added.
human element
Ultimately, the most difficult part of operating agent AI involves the human element. Rose described traditional human work as a “container of work,” a mix of hygiene, productivity, coordination, and specialized tasks. Although an agent cannot run an entire job, it is highly capable of performing specific types of work within its container.
Dell itself was auditing 6,400 jobs across its company to see how its AI agents were impacting its employees.
“The first thing we realized was that every job in the company was changing,” Rose said. “I’m taking jobs out of jobs and taking things out of containers. If the container is only half full, do I need half as many people or do I expand by half? Can I do more specialized work?”
In fact, the impact of AI on the workplace is so significant that change management has become a critical role for IT leaders.
“For the past four months, I have spent 50% of my time dealing with human dynamics,” Rose said. “AI is not a technology or an ROI. [return on investment] discussion. It has now become a dynamic organizational and human discussion. You can’t use these things unless you fully understand how to adapt the human population around you. ”
