Researchers at the University of California, Santa Cruz have devised a way to drastically reduce the energy cost of running large language models.
This is a development that could have a major impact on the use of artificial intelligence (AI) in e-commerce: the approach could significantly reduce power consumption and make advanced AI capabilities more accessible and affordable for businesses of all sizes.
“We achieved the same performance for much less cost. All we had to do was fundamentally change how the neural network worked,” Jason Eshraghian, assistant professor of electrical and computer engineering at UC Santa Cruz's Baskin School of Engineering and lead author of the study, said in a news release on Thursday (June 20). “Then we went a step further and built custom hardware.”
The Cost of AI in E-Commerce
Currently, running advanced AI models like ChatGPT is expensive – recent estimates put OpenAI's energy costs alone at roughly $700,000 per day – and these costs are passed on to the price, creating a significant barrier for small and medium-sized businesses looking to leverage AI in their e-commerce operations.
Research from a team at the University of California, Santa Cruz aims to address the high energy costs associated with running advanced AI models. By eliminating matrix multiplications, the most computationally expensive element of running large language models, they were able to make the models more energy efficient.
“Neural networks are, in a sense, improved matrix multiplication machines,” Eshraghian says. “The bigger the matrix, the more the neural network can learn.”
The researchers claim that their approach is surprisingly effective.
“We were able to run a billion-parameter language model on just 13 watts, roughly the same as powering a light bulb, and more than 50 times more efficient than commodity hardware,” Eshraghian said.
This level of efficiency will enable ecommerce platforms to offer advanced AI-driven features such as personalized recommendations, chatbots, and dynamic pricing at a fraction of the current cost.
Impact on Mobile E-Commerce
The team's innovation also has big implications for mobile e-commerce: “We replaced an expensive operation with a cheaper one,” Rui-Jie Zhu, lead author of the paper and a graduate student in Eshraghian's group, said in a news release.
The reduction in computational complexity achieved by the University of California, Santa Cruz team could make it possible to run full-scale AI models on smartphones. The advancement comes at a time when mobile shopping is growing rapidly.
Once implemented, this technology could significantly enhance mobile shopping experiences and app-based e-commerce by allowing more sophisticated AI-driven features, such as personalized recommendations and advanced search capabilities, to run directly on users’ devices.
Building on their software advances, the team expanded their research by collaborating with other UC Santa Cruz faculty to develop custom hardware designed to maximize the efficiency gains of their new approach.
“These numbers are already very solid, but it's very easy to improve on them,” Eshraghian says. “If we can do this at 13 watts, imagine what we can do with the compute power of an entire data center. We have all this resource, let's use it effectively.”
For e-commerce giants with huge data centers, this could mean significant cost savings and improved AI capabilities, while for smaller businesses, it could ensure a level playing field so they can compete with more sophisticated AI-driven strategies.
As PYMNTS has previously reported, major tech companies such as Microsoft and Google are struggling to monetize generative AI products due to high production, development and training costs.
As the e-commerce industry continues to evolve, innovations like this have the potential to change the way businesses interact with customers, manage inventory, and make strategic decisions. Though the technology is still in its early stages, its potential to democratize advanced AI capabilities in the e-commerce space is enormous.
The researchers have open-sourced their model, which could spur adoption and further innovation in the field. “We've fundamentally changed how neural networks work,” Eshraghian said. The e-commerce industry is watching to see how this shift translates into real-world applications and competitive advantage in digital marketplaces.
