Rakuten AI chief departs from big tech in favor of lower costs

Rakuten Group is expanding its AI team, led by a Google veteran, and building models with a focus on cost efficiency.

Ting Kai, who has been leading the e-commerce pioneer's artificial intelligence team for three years, is tasked with creating AI systems that power the company's many businesses and help process commercial transactions at minimal cost. He oversees a team that has grown to 1,000 people this year and uses “thousands” of Nvidia chips as batteries.

Tokyo-based Rakuten is grappling with a struggling mobile business and constant competition in online shopping, both of which could receive a major boost from effective implementation of new AI tools. But its focus on making money early on is what sets it apart from other big tech companies.

“Rakuten is very business-focused and applies the latest technology to solve customer problems,” Tsai, 53, said in an interview. “To do this at scale, we need to provide the highest margins, so reducing costs in deploying generative AI is very important to us.”

Last week, Cai's team announced version 3 of the company's large-scale language model, which it says is 90% cheaper to operate than its existing LLM counterpart.

The company breaks down tasks into simpler jobs and develops smaller models that address the specific needs of each service. In version 3, approximately 40 billion parameters will be activated for each individual token within the 700 billion parameters, while the remaining parameters will remain inactive for efficiency.

The AI ​​capabilities will contribute 10.5 billion yen (S$86.2 million) to operating profit in 2024, and the company aims to double this figure this year.

Intelligent ad targeting and placement has increased return on investment for sellers using Rakuten's online storefront, and AI-powered semantic search and personalized recommendations have increased user engagement and click-through rates.

“We have observed that users come back more often after using Rakuten AI Marketplace,” Tsai said, referring to the official name of Rakuten's virtual shopping mall. “So what we need to do is reduce the cost of conversations even further. We want to make sure that every time we make a purchase through a conversation, we're making money.”

Related items

Before joining Rakuten, Mr. Tsai worked for a long time as a software engineer at Microsoft, and then at Alphabet, where he worked on Google Maps and local search services. Tsai didn't know much about Rakuten when she was approached by an executive at a Japanese company for a job.

At first, the company was a “strange company” for Tsai, but after getting to know founder and CEO Hiroshi Mikitani and having several conversations with him, Tsai realized they had a lot in common, he said.

The job turned out to be far more than what Tsai believed he had signed up for, he said. His role has grown to be pivotal for companies working to integrate AI across a wide range of services, from mobile assistants to self-driving delivery vehicles.

He is proud of the work his team has accomplished building from the ground up, and Rakuten's long-term vision includes empowering enterprise customers to take advantage of the company's AI tools and expertise.

“A lot of people already rely on Rakuten, but I think we can do a lot more than that,” Tsai said. “Rakuten’s goal is to become Japan’s leading AI empowerment company.”Bloomberg