The rush of integrating artificial intelligence (AI) into enterprise operations clashes with a complex and sometimes underrated reality. Deploying AI at large scale can be expensive, and the real cost can well exceed the rate per million tokens of vendor websites.
Recent PYMNTS intelligence data shows that AI deployment costs are the second biggest drawback of generating AI adoption, with 46.7% citing it as a concern, following only the complexity of integration.
On paper, the cost of using today's generative models is declining based on what AI companies are charging.
For example, Openai's GPT-4, which has an 8K context window, had $30 per million output tokens in early 2023, and $60 per million output tokens.
According to Stanford's 2025 Artificial Intelligence Report, as AI models become more capable and smaller, the cost of applying them in “inference” “falls between 9 and 900 times a year,” the report says.
In terms of infrastructure, annual costs drop by 30% per year, while energy efficiency increases by 40% each year. Additionally, open weight models that are freely available fill the gap with closed performance models.
However, these heading numbers only show part of the story.
Although model costs have been declining since 2022, the overall cost of ownership is “resistant to a decline,” said Muath Judy, founder of SearchQ.ai. “The actual costs lie in hidden infrastructure, including data engineering teams, security compliance, constant model monitoring, and the integrated architects needed to connect AI to existing systems.”
For every dollar spent on AI models, companies spend between $5-10 to make the model “production-ready and enterprise-compliant.” “The challenges of integration tend to be more expensive than the technology itself, and require heavy investments in change management and process redesign, which many organizations underestimate.”
Additionally, the cost of AI deployments is “not a one-off cost, it's an ongoing operational commitment,” Judy added.
So why is AI adoption soaring? “Companies that employ AI aren't waiting for further costs to drop. They're identifying specific use cases that can provide measurable ROIs even at current costs,” Judy said.
Read again: High impact and great reward: Meet CFOs focused on genai
Self-hosting can reduce costs
Pavel Bansevich, Project Manager and Solutions Advisor at Pynest, said many companies could be able to determine 40% of AI costs, including self-hosting, using the cloud, and using third-party infrastructure. Cloud-based hosting may be best suited to prototypes, but costs can skyrocket as workloads scale.
Bantsevich said it collaborated with a US construction company that has been doing business for a century to develop AI predictive analytics tools and hosted it in the cloud. Infrastructure costs have now fallen below $200 per month. But once it gets live and people start using it, the cost surges to around $10,000 a month. Switching to self-hosting using Meta's open-source Llama model instead of the cloud led to a cost reduced to around $7,000 a month, leaving me under control.
In another case, a client of Bansevich's European retailer, with over 50,000 employees, wanted to implement a computer vision module for a self-checkout machine. However, the company did not want to use the cloud. Instead I self-hosted using a small llama ai model that worked well. The cost has now been less than $10 per machine per month. “If the cloud solution had been chosen, the numbers would have been in the air,” he said.
Bantsevich believes that costs will continue to decline as datasets are more readily available today. “AI costs will likely be similar to electricity bills in the near future,” he predicted.
Meanwhile, Bill's Chief Financial Officer Rohini Jain advised businesses to use AI already built into AI for billing, payments and forecasting, rather than adding standalone tools for “uncertain” pricing. “Integrated solutions typically offer more predictable costs, including better ROI and subscription pricing,” she said.
Fergal Glynn, CMO and AI Security Advocate of Mindgard said that AI deployments could cost just $10,000 for a basic project, and large enterprise systems could reach millions of dollars. Most companies spend between $50,000 and $500,000 on real use cases such as analytics tools and chatbots. Small businesses often reduce their payments by using ready-made AI.
Nicole Dinicola, Vice President of Marketing at SmartCat, told PYMNT that hiring AI doesn't have to be “all or nothing.”
“Many platforms, including free or low-cost options, allow organizations to start small and easily expand adoption over time,” Dinicola said. “Unlike legacy SaaS, which requires long-term onboarding, upfront costs and full-scale deployment to demonstrate value, AI can have meaningful impacts without being fully integrated across the organization.”
Dinicola pointed to teams who embed AI into their workflows and already gained efficiency and cost savings. “AI tends to have a worse value, but even small adoptions can drive clear, measurable improvements.”
The bad outcome is that the cost and complexity of AI scares the business to avoid AI deployment in the first place.
“Operation is a more expensive path, even if it's so obvious in advance,” added Dinikola. “Those delays may seem safe, but early adopters are already building momentum, improving their processes, learning faster and expanding their competitive advantage.”
read more:
How to choose to deploy an AI chatbot or agent
Small Business, Big AI: How SMB levelles enterprise giants and arenas
Accounting Services AI may level the playing field for small and medium-sized businesses
