Stay informed with free updates
Just sign up for artificial intelligence myFT Digest — delivered straight to your inbox.
Artificial intelligence companies that have spent billions of dollars building so-called large-scale language models that power their generative AI products are now looking to small-scale language models as a new way to drive revenue.
Apple, Microsoft, Meta, and Google have all recently released new AI models with fewer “parameters” (the number of variables used to train an AI system and shape its output) but with more powerful features. Did.
The move is an effort by technology groups to encourage AI adoption by companies concerned about the cost and computational power required to run large language models, the technology behind popular chatbots such as OpenAI's ChatGPT.
In general, the higher the number of parameters, the better the AI software will perform, and the more complex and nuanced its tasks will be. OpenAI's latest model GPT-4o announced this week and Google's Gemini 1.5 Pro are both estimated to have over 100 billion parameters, and Meta trains his 400 billion parameter version of the open source Llama model. doing.
Not only are some enterprise customers struggling to convince them to pay the hefty costs required to run generative AI products, but there are also concerns that data and copyright responsibilities are hindering adoption. .
That's why technology groups like Meta and Google are looking at just a few billion yen as a cheap, energy-efficient, and customizable alternative that requires less power to train and run, and can even ring-fence sensitive data. Now we're pitching a small language model with parameters.
“Achieving this level of quality at such a low cost allows customers to do things in more applications where the return on investment wasn't prohibitively high enough to justify actually doing it. ” says Eric Boyd. , corporate vice president of Microsoft's Azure AI Platform, which sells AI models to enterprises.
Google, Meta, Microsoft, and French startup Mistral have also released smaller language models that can demonstrate advanced features and focus on specific applications.
Nick Clegg, president of global affairs at Meta, said Llama 3's new 8 billion parameter model is comparable to GPT-4. “I think we're seeing excellent performance on almost every measure you can think of,” he said. Microsoft said its Phi-3-small model, with 7 billion parameters, outperformed an earlier version of the OpenAI model, GPT-3.5.
Smaller models can process tasks locally on the device rather than sending information to the cloud, potentially appealing to privacy-conscious customers who want to ensure that information remains within their internal network.
Charlotte Marshall, managing associate at law firm Addleshaw Goddard, which advises banks, said this is “one of the challenges that I think a lot of our clients are facing” when implementing generative AI products. said it was to comply with regulatory requirements regarding data processing and transfer. He said smaller models offer “an opportunity for companies to overcome” legal and cost concerns.
Smaller models can also run AI functions on devices such as mobile phones. Google's “Gemini Nano” model is built into the company's latest Pixel smartphone and Samsung's latest S24 smartphone.
Apple hinted that it is also developing an AI model to run on its best-selling iPhone. Last month, the Silicon Valley giant released his OpenELM model, a smaller model designed to perform text-based tasks.
Microsoft's Boyd said smaller models would lead to “interesting applications ranging from mobile phones to laptops.”
Sam Altman, head of OpenAI, said in November that the San Francisco-based startup offers customers AI models of various sizes that “serve different purposes,” and will continue to build and build on these options. He said he would continue selling it.
“There are some things that smaller models work very well for,” he added. “I’m really looking forward to it.”
But Altman said OpenAI will continue to build large-scale AI models with scaled-up capabilities, including the ability to reason, plan, and execute tasks, and ultimately achieve human-level intelligence. He added that the focus will be on construction.
“I often think people just want the best model,” he said. “I think that’s what people want most.”
Additional reporting by George Hammond in San Francisco
