According to one study, private AI models will drive 70% of the revenue generated by AI within five years. Recent blog posts From Forrester CEO George Colony.
“Returns will definitely be seen in the private model, because it’s the private model that companies need to differentiate and monetize their own data, not because the private model is inherently better,” Ha Hoang, CIO of cyber resilience company Commvault, told InformationWeek.
5 years is a long time The fast-paced world of AI — New models, new features, and new predictions come out every day. To realize a future where private models represent a significant portion of revenue, CIOs must understand the role that public and private models play within their organizations and adjust their AI roadmaps accordingly.
Why companies build private AI models
There’s a reason why large-scale public AI models are at the forefront of the industry.
“We want access to what the public model is really good at: speed, innovation, and cutting-edge capabilities that allow us to experiment, move quickly, and actually bring new experiences to our users and customers,” Huang said.
However, the public model is not the answer to leveraging a company’s valuable proprietary data. Companies cannot risk exposing confidential internal information. Instead, it can leverage private models to differentiate its internal data and create unique value.
Hoang told Commvault that the question fueling the company’s AI plans is “How do we deliver new capabilities to our customers through our products using private models with their own data and workflows?”
Colony emphasized that idea in his blog, arguing that the goal of AI innovation should be the customer.
“The real AI game will be acquiring, serving, and retaining customers, and that will be the sweet spot for private model business models,” he wrote.
How hybrid AI architectures are changing enterprise IT
Playing that game would require significant changes. Companies need to consider their AI strategy and where their money is going. How much money has been spent on public models? Where should private models be built and invested?
“What we’re really seeing is not a shift away from public models per se, but towards capturing the last mile, where private models tend to be closer to proprietary data, workflows, and outcomes,” Huang said.
Companies will continue to rely on public models to reach that last mile. They will build a private model on a public foundation.
“Right now, we’re using a public model and using RAG to feed it with the latest data that helps differentiate some of the decisions we’re making. We’ve also started fine-tuning the model as it relates to agent deployment,” said Shannon Bell, executive vice president, chief digital officer and CIO at OpenText.
More companies are generating search extensions (rug) Building private models can change the role public models play in the AI ecosystem.
“I think the public model will move upstream and become more like basic infrastructure,” Huang said. “The competition there is who has the best base intelligence at the lowest cost.”
The shift to private and hybrid AI strategies also changes decisions about where and how models are run. As part of this transition, CIOs will also consider where to run their AI models: locally or in the cloud.
“We’re seeing a shift to moving some of that load out of the cloud, especially for simple tasks that can actually be done locally. This gives you more bandwidth and allows you to focus on higher-value tasks, tasks that require more complexity to perform,” said Sebastien Jean, CTO of Phison US.
CIOs, CTOs, and other technology executives are struggling to keep pace with these changes and guide their organizations toward the yet-to-be-determined future of enterprise AI.
Michael Facemire, CTO at Forrester, explained that his days are focused on making sure everything is up and running within his organization and spending time after work to stay on top of the ever-changing landscape of AI. “This is a pace I’ve never seen in my career,” he said.
These leaders cannot realistically keep up with the pace of change on their own. We need help understanding where AI is going and how companies can keep up.
“You need a small wayfinding team to explore the different solutions available to you to determine whether it makes sense to move your organization that way,” Jean says.
What CIOs should consider before building a private AI model
The future of enterprise AI is not a binary question of public or private. This is likely to be a hybrid approach defined by your specific use case.
“We strongly believe that there will be an evolution of hybrid agent models and hybrid cloud architectures where there will be protected private data sets,” OpenText’s Bell said. “And of the two, we have the ability to run agents.”
She added that she expects the adoption of small language models to increase in the coming years, especially in regulated industries.
Facemire points out that cost and energy constraints will also determine the future of enterprise AI.
“We live in a cost-constrained world. You can’t just have a big LLM that does everything for everyone,” he said.
As compute constraints continue, Facemire expects large-scale public models to increasingly lean towards use cases that benefit the bottom line. CIOs will be asked to determine what this means for their strategy.
“They need to understand the individual model and what that model is tuned to work best with,” he said, and then decide what workloads to give them.
What CIOs should consider before implementing private AI
What does all this mean for CIOs building their AI roadmaps today?
“The biggest mistake right now is treating private and public as a technology decision, because I don’t think it’s really that. It’s a value and operating model decision,” Huang said.
CIOs debating how to leverage public versus private models should consider the following:
-
Data preparation and governance. When your AI project stalls and can’t scaleoften it comes down to the data. CIOs want to leverage public and private models, but need a strong foundation of data preparation and governance.
”[Strengthen] data layer, data lineage, metadata data governance, etc. [ensure] Flexibility with workloads and datasets gives you control over your AI strategy as the market continues to evolve. ”
-
Proper use case. not all Use case I need to feed sensitive data into a private model. CIOs must decide when a public model is sufficient and when a private model justifies the cost.
“Private models can certainly require more investment, more governance, more discipline,” Bell said. “It’s important to note that this is why it should be used when there is clear business value and management requirements to justify it.”
-
Vendor flexibility. CIOs want to live in a world where they can take advantage of the new capabilities of AI. This means that Build flexibility into your strategy.
”[Put] With abstraction layers in place, you don’t get locked into one vendor or have to re-architect everything when things change,” said Hoang.
-
talent pipeline. As companies quickly commercialize AI models, they run the risk of accumulating technical debt. without human talent In order to address that technical debt, businesses can face significant bugs and security issues, according to Jean.
“Those who don’t immediately lay off employees or hire junior employees to replace departing senior employees will find themselves in a situation where they don’t get the support they need for the price they’re willing to pay,” he said.
-
costs and outcomes. CIOs are already under pressure to prove that AI delivers measurable value. Going forward, you will need a way to track token costs, administrative overhead, and actual business outcomes.
without it Stronger observability Given cost and performance considerations, Facemire cautioned that CIOs may have a hard time determining whether private AI deployments are actually delivering value.
“Be careful not to turn your service delivery model upside down,” Feithmeyer says. “If you wait too long and don’t build an observability layer into your own private model now, you may realize it too late.”
