Celestial AI Raises $100M to Expand Photonic Fabric Technology Platform

Optical interconnect technology developer Celestial AI has announced a successful Series B funding round, raising $100 million for its Photonic Fabric technology platform. IAG Capital Partners, Koch Disruptive Technologies (KDT) and Temasek’s Xora Innovation Fund led the investment.

Other participants include Samsung Catalyst, Smart Global Holdings (SGH), Porsche Automobile Holding SE, The Engine Fund, ImecXpand, M Ventures and Tyche Partners.

According to Celestial AI, the company’s Photonic Fabric platform represents significant advances in optical connectivity performance, outperforming existing technologies. The company has raised a total of $165 million in seed funding through Series B.

Tackle the “memory wall” challenge

Advanced artificial intelligence (AI) models such as ChatGPT and GPT-4, which is widely used in recommendation engines, require exponential increases in memory capacity and bandwidth. However, cloud service providers (CSPs) and hyperscale data centers are challenged by memory scaling and computing interdependencies, commonly referred to as the “memory wall” challenge.

Electrical interconnect limitations such as limited bandwidth, high latency, and high power consumption hinder the growth of AI business models and advances in AI.

To address these challenges, Celestial AI collaborated with hyperscalers, AI computing and memory providers to develop a photonic fabric. Optical interconnects are designed for distributed exascale computing and memory clusters.

The company claims its proprietary Optical Computing Interconnect (OCI) technology enables scalable data center memory isolation, enabling faster computing.

Problems where memory capacity matters

Dave Lazovsky, CEO of Celestial AI, told VentureBeat, “Going forward, the main issue will be memory capacity, bandwidth, and data movement (interconnect between chips) in large scale language model (LLM) and recommendation engine workloads. ).Our photonic fabric technology allows us to integrate photonics directly onto the silicon die.The main advantage is that our solution allows us to deliver data to computing points at any point on the silicon die. Competing solutions such as Co-Packaged Optics (CPO) simply deliver data to the edge of the die and cannot do this.”

Lazovsky argues that photonic fabrics have successfully addressed the coastal challenge by providing significantly increased bandwidth (1.8 Tbps/mm²) at nanosecond delays. As a result, this platform provides a fully photonic inter-computing and compute-to-memory link.

The recent funding round has also caught the attention of Broadcom, which is working with them to develop photonic fabric prototypes based on Celestial AI designs. The company expects these prototypes to be ready for shipment to customers within the next 18 months.

Realization of high-speed computing by optical interconnect

Lazovsky said that as the amount of data transferred within a data center increases, so should the data rate. As these speeds increase, he said, electrical interconnects face problems such as loss of signal fidelity and limited bandwidth that cannot scale with increasing data, thus limiting overall system throughput. I explained that I would.

According to Celestial AI, photonic fabric’s low-latency data transmission makes it easier to connect and disconnect many more servers than traditional electrical interconnections. This low latency allows latency sensitive applications to take advantage of remote memory. This is a possibility previously unattainable with conventional electrical interconnections.

“We enable hyperscalers and data centers to distribute memory and compute resources without sacrificing power, latency or performance,” Lazovsky told VentureBeat. “Inefficient use of server DRAM memory leads to wasted hundreds of millions (if not billions) of dollars across hyperscalers and enterprises. , can not only reduce memory usage, but also prove memory utilization.”

Store and process larger datasets

The company claims its new product can deliver data directly from any point on the silicon to the computing point. According to Celestial AI, the photonic fabric exceeds the limits of silicon edge connectivity, offering a package bandwidth of 1.8 Tbps/mm², 25 times the bandwidth offered by CPO. What’s more, by delivering data directly to computing points rather than the edge, Photonic Fabric achieves 10 times lower latency than his, the company claims.

Celestial AI aims to simplify enterprise computing for LLMs such as GPT-4, PaLM, and Deep Learning Recommendation Models (DLRM). These sizes can range from 100 billion to over 1 trillion parameters.

Lazovsky says AI processors (GPUs, ASICs) have a limited amount of high-bandwidth memory (32GB to 128GB), so companies today are deploying hundreds to several of these processors to handle these models. I explained that I need to connect a thousand pieces. However, this approach reduces system efficiency and increases costs.

“By increasing the addressable memory capacity of each processor at high bandwidth, the photonic fabric allows each processor to store and process larger chunks of data, reducing the number of processors required.” he added. “By providing fast chip-to-chip links, connected processors can process models faster, increasing throughput while reducing costs.”

What’s next for Celestial AI?

Lazovsky said the funding from this round will be used to accelerate the productization and commercialization of the Photonic Fabric technology platform by expanding Celestial AI’s engineering, sales and technical marketing teams. .

“Given the increase in LLM-generated AI workloads and the pressure it exerts on current data center architectures, the demand for optical connectivity to support the transition from general computing data center infrastructure to high-speed computing is rapidly increasing. ,” Lazovsky told VentureBeat. “By the end of 2023, we expect to increase our headcount by approximately 30% to 130 employees.”

He said that as the use of LLM expands to a variety of applications, infrastructure costs will increase proportionately, resulting in negative margins for many internet-scale software applications. Additionally, data centers are reaching power limits, limiting the amount of computing they can add.

To address these challenges, Lazovsky aims to minimize reliance on expensive processors by providing high-bandwidth, low-latency chip-to-chip and chip-to-memory interconnect solutions. increase. He said the approach aims to reduce companies’ capital expenditures and increase the efficiency of their existing infrastructure.

“By breaking down memory barriers and helping improve system efficiency, we aim to help shape the future direction of AI model advancement and deployment through new products,” he said. “Once memory capacity and bandwidth are no longer limiting factors, data scientists can experiment with larger and different model architectures to explore new applications and use cases. We believe that by reducing , more companies and applications will be able to adopt LLM more quickly.”