interview Naveen Rao founded an AI business and sold it to Intel and Databricks. He is now turning his attention to satisfying his thirst for the power of AI, and believes his new company, Unconventional AI, can do just that by building chips inspired by nature.
To answer that question, Rao revealed on Monday that Unconventional AI has raised $475 million in seed funding from Jeff Bezos, Andreessen Horowitz, Lightspeed and others.
“AI is inherently tied to hardware, and hardware is inherently tied to power. Because of energy issues, it cannot scale beyond a certain number of inferences per unit time. We cannot produce any more energy in the next 10 years,” Rao said. register.
When it comes to unconventional AI, Rao argues that we are using the wrong tools for the job.
“Natural learning systems never used numbers. They never simulated the dynamics of learning. They use the inherent physics of whatever substrate they are on to build a learning system,” Rao said. “We believe we can reproduce that behavior in silicon.”
Mr. Rao is no stranger to this concept. Before founding MosaicML and Nervana Systems, Mr. Rao was acquired by Databricks and Intel, respectively. He studied electrical engineering at Stanford and received his PhD in neuroscience from Brown University.
The idea that biological systems, shaped by millions of years of evolution, may provide clues to more efficient computer architectures is not new. Companies like IBM and Intel have been chasing it for years. If our brains can operate on just 20 watts of bioelectrical energy, imagine what we could do with megawatts, never mind the gigawatt-sized data centers currently being built.
This class of computers is known as “neuromorphic,” and their designers aim to reverse engineer the brain's inner workings and implement them in silicon. Despite decades of research in this field, only a handful of working prototypes have been created. Lowly creatures like owls aside, nothing can even remotely come close to the performance and efficiency of the human brain.
Just because progress is slow doesn't mean this approach is wrong. “Some of these things don't work until they actually work. Until the mid-2000s, neural networks were considered kind of a backwater,” Rao says. Things have changed as computing has become more abundant.
Unconventional AI isn't just focused on neuromorphic computing. “The problem with neuromorphics is that it has to function like a brain. But why does it have to function like a brain?” Rao says. “Perhaps there are concepts from the brain that can help build such systems. [learning] system. That's the way we see it. It doesn't mean it has to function like a brain. ”
Instead, Unconventional AI's lab is considering several different approaches to improve the efficiency of machine learning accelerators, Rao said. He declined to provide details of the company's research, but what is known is that they will likely be made of silicon and will be analog chips rather than digital devices.
“These are nonlinear dynamics of a circuit. It's analog in nature,” he said. “All devices are analog, even 'digital' devices. We're just designing those circuits to work digitally, but by making those circuits 1s and 0s, we're largely erasing the rich functionality that those circuits can perform.”
For many computational workloads, the determinism provided by digital systems is desirable. For example, you don't want accounting software that spits out different answers every time.
However, machine learning is often non-deterministic in nature and does not necessarily require a deterministic computing platform. Rao envisions scenarios where a combination of non-deterministic analog logic and deterministic digital logic is used to accelerate various aspects of machine learning workloads.
Rao said certain models are better suited to the kind of nonlinear dynamics that Unconventional is targeting. “Diffusion models, flow models, energy-based models, etc. are dynamic in nature,” he said.
The CEO believes this problem will take time to resolve.
“The product won't be ready within two years,” he said. “This is primarily a research effort over the next few years, and we are serious about breaking new paradigms.”
That said, Rao plans to share some unconventional AI discoveries along the way, perhaps as soon as next year. “This is not something we study in a lab for four years and come up with a solution,” he said. “We plan to start rolling it out over the next few months.”
Although Rao's initial focus is research, his long-term goal is to start a systems company. ®
