TL;DR:
- Japan trails in the generative AI race due to shortcomings in deep learning and extensive software development.
- Large language models (LLMs) are key to generative AI, but Japan lags behind the US, China, and the EU in developing algorithms.
- A deficit of software engineers and a lack of world-class AI supercomputers pose challenges for Japan.
- Government-controlled supercomputers like Fugaku offer hope, with plans to introduce a new cutting-edge supercomputer by 2024.
- Collaboration between institutions and significant investments in Hokkaido’s supercomputer emphasize Japan’s commitment to closing the gap in generative AI.
Main AI News:
In the fast-paced realm of artificial intelligence (AI), Japan finds itself striving to catch up in the generative AI race, with countries like the United States, China, and the European Union leading the charge. Japan’s lag is largely attributed to its shortcomings in deep learning and extensive software development, which are pivotal in the development of large language models (LLMs) that fuel AI innovations like ChatGPT and Baidu’s Ernie Bot.
To delve deeper into generative AI, large language models play a crucial role as they enable the processing of vast datasets to generate text and other content. However, Japan trails behind its global counterparts in the development of these algorithms. Noriyuki Kojima, the co-founder of Japanese LLM startup Kotoba Technology, highlighted Japan’s situation in a recent interview with CNBC. According to a report by Reuters, Chinese organizations, including tech giants Alibaba and Tencent, have domestically launched at least 79 LLMs over the past three years, emphasizing the competitive landscape Japan faces.
The United States boasts corporate powerhouses like OpenAI, Microsoft, Google, and Meta, which significantly contribute to the country’s LLM advancements. Deep learning, in particular, requires a robust community of software engineers to develop the necessary infrastructure and applications, adding further complexity to Japan’s challenge. The country is predicted to face a deficit of 789,000 software engineers by 2030, according to Japan’s Ministry of Economy, Trade, and Industry (METI). Moreover, Japan grapples with hardware challenges, lacking “world-class machines” like IBM’s Vela and Microsoft’s Azure-hosted system, essential for training LLMs.
Nonetheless, there is hope on the horizon for Japan’s pursuit of generative AI. Government-controlled supercomputers, such as Fugaku, hold the key to unlocking the potential of LLMs in Japan. Access to such large-scale supercomputers is critical in overcoming the bottlenecks in the development process, as stated by Kojima. Encouragingly, METI has plans to introduce a state-of-the-art supercomputer through its affiliated laboratory, aiming to support the development of generative AI in Japan. Nikkei’s latest report indicates that the National Institute of Advanced Industrial Science and Technology (AIST) is working on a supercomputer with computing capability 2.5 times greater than its existing machine, slated for completion as early as 2024.
Under METI’s supervision, the research arm will offer this cutting-edge supercomputer to domestic companies developing generative AI through a cloud service. To support these efforts, AIST plans to establish a new research center for supercomputers and quantum technologies, funded by a generous 32 billion yen (US$226 million) from METI. Furthermore, AIST’s existing supercomputer, accessible through a cloud service, will be made available to Japanese companies. Additionally, other notable institutions, such as the Tokyo Institute of Technology and Tohoku University, are collaborating on developing LLMs based on Japanese data using Fugaku supercomputer, with results to be published in 2024.
Beyond institutional collaborations, the Japanese government is actively investing in the development of generative AI. With a plan to invest 6.8 billion yen (US$48.2 million), approximately half the total cost, a new supercomputer in Hokkaido will be built and operational as early as 2024. This specialized supercomputer will focus on LLM training, fostering Japan’s progress in generative AI. Notably, the supercomputer’s assembly in Ishikari City, led by cloud service provider Sakura Internet, is equipped with over 2,000 graphics processing units from Nvidia, crucial components for LLMs. It is projected that Sakura Internet’s supercomputer can develop GPT-3 in just three days and GPT-4 within a year.
Conclusion:
Japan’s dedication to investing in cutting-edge supercomputers and fostering collaborations between institutions reflects its commitment to closing the gap in the generative AI market. By addressing its shortcomings in deep learning and software development and harnessing the power of large language models, Japan stands to make significant strides in AI innovation. As a result, the market can expect Japan to become a more formidable player in the global AI landscape, potentially redefining industries and contributing to groundbreaking advancements in generative AI technology.
