Japan is known for its futuristic technology. But the country is lagging behind in the generative AI race and is looking to create its own large-scale language models.
Cole Photographer | Moments | Getty Images
Countries are racing to develop their own generative artificial intelligence algorithms, but Japan is already lagging behind in the high-tech sector.
Generative AI has become one of the trendiest topics in tech since OpenAI made headlines with its chatbot ChatGPT. Breakthroughs in generative AI have the potential to increase global GDP by 7%, or nearly $7 trillion, over the next decade, according to a Goldman Sachs study.
Key to generative AI development are large-scale language models, such as ChatGPT and Baidu’s Ernie Bot, that can process massive data sets to generate text and other content. But Japan currently lags behind the US, China and the EU in developing such algorithms, says Noriyuki Kojima, co-founder of Japanese LLM startup Kotoba Technology.
Chinese organizations including tech giants Alibaba and Tencent, At least 79 LLMs have been launched in the country in the past three years, Reuters reported in May, citing findings from a consortium of state-run laboratories. US powerhouses such as OpenAI, Microsoft, Google and Meta are playing a key role in driving his LLM progress in the country, Kojima said.
However, Japan lags behind the US, China and Europe in terms of scale and speed of LLM development.
“Japan’s latecomer position in the field of generative AI is largely due to its comparative shortcomings in deep learning and broader software development,” Kojima said.
Kojima added that deep learning requires a “strong community of software engineers” to develop the necessary infrastructure and applications. However, according to the Ministry of Economy, Trade and Industry, Japan will be short of 789,000 software engineers by 2030.The country is currently ranked 28thth According to the IMD Global Digital Competitiveness Ranking, it ranks first out of 63 countries in terms of technical knowledge.
Japan also faces hardware challenges as LLMs need to be trained using AI supercomputers such as IBM’s Vela and Microsoft’s Azure-hosted systems. But no Japanese private company has its own “world-class machine” with such capabilities, Nikkei Asia reported.
Government-controlled supercomputers like Fugaku are therefore “key” to Japan’s LLM pursuit, Kojima explained.
“Access to such massive supercomputers has traditionally been the biggest bottleneck in the process, and thus forms the backbone of LLM development,” he said.
Tokyo Institute of Technology and Tohoku University plan to work with supercomputer developers Fujitsu and RIKEN to develop an LLM using Fugaku based primarily on Japanese data, Fujitsu said in May. announced to
The organizations plan to publish their research results in 2024 to help other Japanese researchers and engineers develop the LLM, Fujitsu added.
In addition, the Japanese government will invest 6.8 billion yen ($48.2 million), about half of the total cost, to build a new supercomputer in Hokkaido, which could start service as early as next year, Nikkei Asia reported. According to Nikkei Asia, the supercomputer will specialize in LLM training to boost Japan’s generative AI development.
In April, Japanese Prime Minister Fumio Kishida said Japan supports the industrial use of generative AI technology. Kishida’s remarks came after a meeting with OpenAI CEO Sam Altman, who said the company was considering opening an office in Japan.
Big tech companies are also joining the fray to boost Japan’s position in generative AI. SoftBank’s mobile division announced in June that it plans to develop its own generative AI platform, local media reported. This was underscored by SoftBank CEO Masayoshi Son’s announcement that the investment firm plans to shift from “defensive mode” to “offensive mode” and increase its focus on AI.
“We would like to be [in] We are well positioned to lead the AI revolution,” Son said at the annual shareholders meeting.
SoftBank Group sold an 85% stake in SB Energy to Toyota Tsusho in April and recently agreed to sell a 90% stake in US investment firm Fortress Investment Group, Nikkei Asia reported. Cutting back on those other investments has freed up cash for SoftBank to focus primarily on AI through the Vision Fund’s venture capital investment arm.
Arm, a chip design company owned by SoftBank, also plans to go public in the U.S. by the end of the year. “This will be the largest IPO ever to hit the world,” said Amir Anvarzadeh, Japanese stock market strategist at Asymmetry Advisors.
The IPO will provide a huge amount of cash to bolster SoftBank, which reported a record loss of 4.3 trillion yen for the Vision Fund in the fiscal year ended March 31.
Arm was originally looking to raise between $8 billion and $10 billion. But with demand for chips “seeing no ceiling,” Arm could raise up to $50 billion to $60 billion, or “85% of SoftBank’s market capitalization,” Ambalzadeh said. suggested.
He said SoftBank’s stock price would likely rise, but that doesn’t guarantee the company’s AI efforts will succeed.
“Fundamentally, I don’t think SoftBank is going to change the situation in Japan…they are not AI saviors in Japan,” he said.
Japanese telecommunications company NTT also announced plans to develop its own LLM this year, with the aim of building a “lightweight and efficient” service for enterprises. NTT has announced that it will invest 8 trillion yen in growth areas such as data centers and AI over the next five years, a 50% increase from previous investments.
Local media reported that digital advertising company CyberAgent released an LLM in May that allows companies to create AI chatbot tools. The company says it’s one of the few “models dedicated to Japanese language and culture.”
Japan is still lagging behind in generative AI, but is taking its first steps with these private sector efforts. Once a “robust infrastructure” is established, the use of open-source software and data from previous pioneers will likely “significantly reduce” the remaining technical challenges, Kojima said. said. Bloom, Falcon and RedPajama are all open source LLMs of his trained on vast amounts of data that can be downloaded and explored.
But companies entering the space should expect competition over a “relatively long period of time,” Kojima said. Developing LLM requires significant capital investment and a workforce highly skilled in natural language processing and high-performance computing, he explained.
“Even if SoftBank and NTT enter this race, the AI landscape will not change in the short term.”
The increasing participation of Japanese technology companies in generative AI development is consistent with their willingness to adopt AI in other fields. A survey by Teikoku Databank found that more than 60% of Japanese companies are positive about using generative AI in their operations, with 9.1% already using it.
Hitachi announced in May that it had established a generative AI center to encourage employees to use technology safely and effectively. The center said it will draw on the expertise of data scientists, AI researchers and relevant experts to formulate guidelines to mitigate the risks of generative AI.
Chief Cabinet Secretary Hirokazu Matsuno said Japan would also consider introducing AI technologies like ChatGPT into the government once cybersecurity and privacy concerns are resolved.
As Japan becomes more open to the use of generative AI, the government will need to develop and promote soft guidelines for its use, while assessing the need for hard regulation based on specific risks, Kyoto said. Hiroki Habuka, a research professor at the university’s Graduate School of Law, said:
“Without clearer guidance on how companies should behave when using generative AI, practice could become fragmented,” he said.
