IBM wants to teach AI the language of business

At VB Transform 2024, IBM's David Cox built on the company's longstanding commitment to open source technology to make a compelling case for open innovation in enterprise generative AI. As vice president of AI Models and director of the MIT-IBM Watson AI Lab, he presented a vision that will challenge and inspire the tech industry.

“Open innovation is really the story of human progress,” said Cox, positioning the concept as a foundation for technological advancement. Cox emphasized the importance of the current situation in AI development, saying, “I think this situation is particularly important because we have to make decisions about where we want to invest. How do we avoid lock-in?”

All kinds of open

The IBM executive emphasized a nuanced view of openness in AI, challenging the idea that it's a simple binary concept. “Open isn't just one thing. It can actually mean many things,” Cox explained. He noted a growing ecosystem of open models from a variety of sources, including tech giants, universities and even nation states.

However, Cox expressed concern about the quality of openness in many law masters programs. “In some cases, you're given what looks like a binary,” he warned. “You're given what looks like a bag of numbers, and you don't know how it was generated.” This lack of transparency could make it difficult or impossible to replicate these models, undermining a key tenet of open source principles, Cox argued.

Drawing parallels to traditional open source software, Cox outlined several characteristics that make these projects successful, including frequent updates, structured release cycles, regular security fixes, and active community contributions. “It doesn't change dramatically from version to version, so everything is well-defined and allows for incremental contributions from within the company and across the community,” he said.

LLM: Open in name only?

Cox then looked at the current state of Open LLM, noting that it lacks many essential open source properties. “Open LLM is a great thing, but today it doesn't have many of those properties,” he noted. He criticized some companies' erratic release patterns, saying, “Companies can release a new generation of models at any time. Some model providers release models and then never go back and release updates.”

Cox argued that this approach falls short of true open source principles and limits the potential for community-driven improvements and innovation in AI. His insights urge the AI ​​industry to reevaluate its practices around the open source model and call for a more standardized, transparent, and collaborative approach to AI development.

To illustrate his point, Cox pointed to IBM's work in this direction with its Granite series of open-source AI models. “We're completely transparent about everything that goes into our models,” Cox explained, emphasizing IBM's commitment to transparency. “We'll tell you exactly what's in the models. We've open-sourced all of our processing code, so you can see exactly what we've done to it to remove objectionable content and filter for quality.”

Cox argued that this level of openness doesn't come at the expense of performance: He presented benchmarks comparing Granite's code models with other leading models, saying, “These are state-of-the-art models. You don't need to have an opaque model to have a performant model.”

Corporate Data Gaps

Cox also offered a new perspective on LLMs, one that sees them primarily as data representations, rather than simply as conversation tools. This shift in perception comes at an important time, with predictions that within the next 5 to 10 years, LLMs will encompass nearly all publicly available information. But Cox noted a major gap: a company's unique “secret sauce” is largely unrepresented in these models.

To address this, Cox proposed a mission to represent enterprise data within a foundational model and maximize its value. Techniques such as search augmentation generation (RAG) are common, but Cox argued that they fall short in leveraging enterprise-specific knowledge, policies, and proprietary information. The key, he argued, is for the LLM to truly understand and incorporate this enterprise-specific context.

Cox outlines a three-step approach for enterprises: find an open, trusted base model; create new representations of business data; and then deploy, extend, and create value. He emphasizes that choosing the base model carefully is crucial, especially in regulated industries. Transparency is crucial. “There's a wide range of characteristics that enterprises need — regulated industries, other industries where you need transparency — and often model providers won't tell you what data is in the model,” Cox says.

The challenge is successfully combining your own data with the base model. To achieve this, Cox argues that the base model you choose needs to meet several criteria. As a baseline requirement, it needs to perform well. And more importantly, it needs to be transparent so that companies can fully understand what's in it. Of course, the model needs to be open source and provide the flexibility and control that companies need.

Teach AI the secrets of your business

Building on his vision of integrating enterprise data with open source LLM, Cox announced InstructLab, a joint IBM and Red Hat project to bring the concept to life. First reported by VentureBeat in May, the effort represents a practical implementation of Cox's three-phase approach to enterprise AI adoption.

InstructLab is addressing the challenge of incorporating unique corporate knowledge into AI models, offering, as Cox explains, “a true open source contribution model for LLMs.”

The project's methodology revolves around a taxonomy of global knowledge and skills, allowing users to precisely target areas of model enhancement. This structured approach facilitates the integration of enterprise “secret sauce,” which Cox noted is currently missing from LLMs. InstructLab lowers the barrier for domain experts to get involved in customizing the model by allowing them to contribute with simple examples and relevant documentation.

InstructLab addresses the challenge of mixing your own data with a base model by using a “teacher” model to generate synthetic training data. This innovative approach allows you to add enterprise-specific features while maintaining model performance.

Notably, InstructLab has significantly accelerated its model update cycle — “we could reverse that in a day,” Cox said, contrasting it with the traditional “monolithic, sort of yearly release cycle.” This agility allows companies to quickly integrate new information and adapt their AI models to changing business needs.

Cox's insights and IBM's InstructLab signal a shift in enterprise AI adoption. The focus is shifting from one-size-fits-all, off-the-shelf models to customized solutions that reflect each company's unique expertise. As the technology matures, companies that can most effectively translate their organizational knowledge into AI-powered insights may gain a competitive advantage. The next chapter of AI isn't just about smarter machines; it's about machines that understand your business as well as you do.