IT leaders can expect a difficult task when scaling generative AI beyond experiments and pilot projects.
Participants at the 2024 MIT Sloan CIO Symposium, which concluded yesterday in Cambridge, Massachusetts, cited several obstacles to enterprise-grade GenAI adoption. This list includes regulatory concerns, data issues, elusive business value, and implementation orchestration challenges.
Aamer Baig, senior partner at McKinsey & Company, used a human metaphor to explain recent advances in technology and the “hard truths” that CIOs must face.
“Like probably many great relationships, it started with a very intense belief in inevitability. [and] “We got validation from a lot of opinion formers around us,” he said in a presentation at the CIO Symposium, adding, “We started having fun with synthesis and content generation, and we started to take that into real-world business problems.” “I started applying it,” he said.
The difficult part is the next step. Baig cited McKinsey's unpublished global AI study, which found that only 11% of companies surveyed had implemented his GenAI at scale.
“There are a number of issues that need to be resolved before we can move from piloting to scale-up,” he says. “It's important to set a framework and say, 'We see the honeymoon period as over.'
Investing in data
These issues include the availability of data and the quality of data underlying the deployment of generative AI. The performance and reliability of such systems depends on the accuracy and relevance of the data. However, companies may be tempted to ignore the data step and rush into implementation.
“We are very excited about the adoption of AI in general and GenAI in particular,” said Anish Atariye, CTO of Cleanlab, a San Francisco-based provider of data curation and quality tools. “However, we are not yet at the point where it is generally known that data quality is an important piece of the puzzle.”
Athalye, whose company participated in the CIO Symposium's Innovation Showcase, said organizations at the forefront of AI understand the importance of data quality. However, the degree of knowledge of the data below that layer varies.
“They may or may not have reached a stage where they realize this is not something they can do on a whim,” he added.
As organizations approach data initiatives, the question becomes project scope. Baig said many organizations are taking a top-down approach to identifying his GenAI use cases and asking for the data needed to support them. As a result, large projects are unlikely to be completed in a reasonable time, he noted.
Baig instead advised organizations to focus on a small number of data domains that can drive multiple high-priority use cases.
“Typically there will be three or four domains that you can actually start,” he said.
The latest stage in the evolution of generative AI will see companies looking to scale the technology.
Track regulatory trends
Even as CIOs delve into GenAI and other technology initiatives, they need to keep an eye on data privacy and security regulations and regulatory compliance obligations. Stuart Madnick, co-founder of the MIT Sloan Research Program in Cybersecurity, said his group is studying more than 170 new cybersecurity regulations impacting IT.
“These regulations and laws are coming from everywhere, from the White House, from Congress, from just about every three-letter institution,” said Madnick, who spoke on cyber resilience at an MIT event.
He noted that in addition to the U.S. federal sector and state governments, the European Union, India, and China are also contributing to the international regulatory landscape. Governments are pursuing AI regulation to address concerns such as consumer protection and intellectual property rights.
Gayatri Shenai, senior partner at McKinsey, said IT leaders need to consider security, privacy and regulatory compliance when choosing a GenAI partner. These factors need to be added to traditional partner evaluation criteria such as financial viability, said Shenai, who moderated a panel discussion on managing multivendor partnerships at the CIO Symposium.
She summarized her questions for IT leaders: “How do we establish awareness of compliance requirements and change compliance requirements? [partner selection] Maybe there were guardrails? ”
Narrow down the use case list
Since GenAI technology became mainstream in late 2022, companies have been brainstorming numerous use cases. The challenge is now focused on those most likely to yield tangible results, industry executives said.
There are many issues to resolve before moving from pilot to scale-up.
Armor BaigMcKinsey & Company Senior Partner
Companies may launch many generative AI projects, but very few of them will generate a bottom line profit, Baig said. He noted that McKinsey's global AI survey found that only 15% of companies see revenue improvements from generative AI.
“One of the most important roles a CIO can play is to focus the organization on efforts that drive real business value: solving critical business problems that are technically feasible,” Baig said. Stated.
As generative AI applications grow and become more expensive, a solid business case becomes increasingly important. Some of these costs may not be immediately obvious, Baig noted. GenAI's low initial cost belies the cost of running and maintaining the system over time, he said.
Beyond technology, change management costs can increase by three times the cost of implementing generative AI, Baig added.
Coordinating your deployment
IT leaders also need to assemble a multi-layered technology stack to deploy generative AI.
“Even a simple generative AI solution requires about 20 to 30 pieces that need to come together,” Baig says.
These elements include user interfaces, data enrichment features, security and access controls, and API gateways that link to the underlying model. Baig said automating technical workflows such as model testing and validation is also critical to realizing the full benefits of generative AI.
Another challenge is integrating GenAI tools with enterprise IT environments that include legacy systems. Conference attendee Darlene Williams, senior vice president and CIO of Rocket Software, said there is a wealth of data on enterprise mainframes that can be used to train AI models.
“I think mainframes will definitely drive AI,” she said, pointing to generative and predictive AI as examples.
Waltham, Massachusetts-based IT modernization software company Rocket Software completed its acquisition of OpenText's mainframe modernization business earlier this month.
Companies embarking on complex GenAI deployment and integration projects will need to coordinate multiple partners as well as a myriad of technologies. This requires greater collaboration among more stakeholders, McKinsey's Shenai said. Those parties may include the Product Owner. AI Safety, Reliability, and Accountability Group. She noted that this includes IT procurement organizations, project managers and program managers.
“Those are pieces that people don’t talk about enough,” Shenai said.
John Moore is a TechTarget editorial writer, covering CIO roles, economic trends, and the IT services industry.
This website uses cookies to ensure that you get the best experience on our website. OkRead More
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are as essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
