That’s a big number.
I don’t know what over 1000 trillion is, but it’s a significant amount.
So does that mean demand means more revenue for AI infrastructure providers, and as costs come down, revenue increases?
correct. We have been observing a decline in the unit price of computing tokens for some time. Semiconductor providers are reducing the cost per token of inference, a process that uses trained LLMs to obtain results, by 60% to 70% annually. That’s a very, very rapid rate of decline. This is happening thanks to increased chip efficiency and new architectural efficiencies in AI data center architectures.
We believe these improving economic conditions are likely to drive gross margin growth over the next three to 12 months. So we’re at an interesting tipping point.
Will AI chip makers be able to keep up with demand?
I think it can be done in the long run. It could take three years to build a new chip factory. Obviously, things are moving faster than that. And if you rewind six months and we’re just talking about chatbots, there’s probably enough capacity to handle that easily.
The problem is that use cases are evolving very quickly. A year ago we weren’t talking about agents, now we are. What’s happening is that the industry is responding to capacity that was needed six months ago. But the goalposts are shifting, and chip manufacturing production systems can’t react as quickly. There will be shortages for the next 12 months or more. I think we can catch up within two years.
Will this pave the way for hyperscalers to improve cash flow performance?
yes. Currently, semiconductor manufacturers in this space have gross margins of over 70%, which is fine for these companies. The problem lies with hyperscalers and how most of their free cash flow is consumed by capital expenditures. This affects gross profit. But a crossover is coming. We are seeing changes in gross margins because costs are falling faster than prices.
What is driving the demand for agent AI in the consumer market?
If you think about it, a lot of consumer activity is about online queries. Much of today is traditional search. By 2030, traditional search is expected to decrease as a proportion of queries and be replaced by things like large-scale language model queries. But here’s an example of how to use an agent.
Smartphone hijacking agents already exist in China and perform various tasks in the background. For example, “Book a flight to Singapore” or “Organize your main inbox and filter all junk emails to organize all your emails into business priorities.”
These are becoming more autonomous in nature. We are entering a phase of “always on” background agents that perform tasks when needed.
So you can imagine that the combination of all these queries will change quite dramatically over the next five years. Modeling shows that daily queries to LLMs will grow at a compound annual growth rate of 40%, reaching 11 billion by 2030.
Why has agent AI in the enterprise sector been slow to grow?
The reason is that applying agent AI to business is more complex. Writing code or software is much more complex than booking a flight to Singapore. Customer service calls are also more complicated.
It doesn’t just work, it needs to be tested, retested, integrated with other code, retested, and documented. It must also work in the context of compliance, rules, budget parameters, and other requirements of the company.
Importantly, adoption rates are still relatively low, especially among small and medium-sized enterprises. It is predicted that 12% of knowledge workers will use agent AI in 2030, and by 2040 that number will be 37%. There is a very long tail of recruitment going on.
Is there a risk that the benefits of increased demand and lower costs will not materialize for all AI providers?
Risk of not seeing margin improvement across all AI workloads. In other words, the adoption of agent AI in the enterprise sector is expected to be uneven. For example, tasks such as coding are highly efficient because agents can start working autonomously and quickly come back independently.
Text-based chatbots, such as customer service agents, are already very efficient. But there are other jobs with technical factors that make them unattractive for agent AI. We found a case for a real-time voice agent where the human costs are actually less than today’s LLM costs due to what we call the “time-sensitivity” and “delay characteristics” of the software. Therefore, the economic situation is not very favorable.
Taking a step back, the important point is that the surge in demand for agent AI may reset assumptions about what’s next for this industry.
yes. I think the shift in profit margins for hyperscalers and model providers is very different from the general market view that the use of AI simply increases the cost burden and makes it unsustainable.
Evolution will be uneven and somewhat non-linear. Not all players are at the same level. We will start to see differentiation between hyperscalers, especially when it comes to operating cash flow. All players are pulled upwards, but at different speeds.
