Former Carnegie Mellon University professors Deepak Pathak (left) and Abhinav Gupta (right) founded Skild AI in 2023 to build a “universal brain” for AI.
Skilled AI
Whether they're bipedal humanoids performing basic factory tasks or quadrupedal military “robot dogs” for urban warfare, robots need a brain. Historically, robots have been highly specialized and purpose-built. But a Pittsburgh-based robotics startup claims to have developed a single, off-the-shelf intelligence that can be embedded into a wide range of robots to perform basic functions.
Founded in May 2023 by former Carnegie Mellon professors Abhinav Gupta and Deepak Pathak, Skild AI has created a basic model of a “general-purpose brain” that can be built into a variety of robots, allowing them to climb steep slopes, walk over objects blocking their path, and identify and pick up items.
The company announced on Tuesday that it had raised $300 million at a valuation of $1.5 billion in a Series A funding round led by Lightspeed Ventures, SoftBank, Coatue and Amazon founder Jeff Bezos, with participation from CRV, Felicis Ventures, Menlo Ventures, Amazon and General Catalyst, among others.
“We're excited to be partnering with NVIDIA to bring this technology to market,” said Raviraj Jain, a partner at Lightspeed who led the company's seed round in July 2023. Forbes He was very impressed with Skild AI's model when he first saw it being pressure tested last April. Using the model, a robot was able to perform tasks in an environment it had never seen before and was not designed for the demo. “The robot was able to climb stairs at that time, and I think it's really impressive that it was able to do that, given that it's a very complex stability problem,” he said.
Even more impressive, robots using Skilled's AI models also demonstrated “emergent capabilities” — entirely new abilities that had not been taught to them. These were often simple, like retrieving an object that had slipped out of a hand or rotating an object. But the models demonstrated the ability to perform unexpected tasks, a trend seen in advanced artificial systems like large-scale language models.
Skilled achieves this by training its models on a massive database of text, images, and videos that the company claims is 1,000 times larger than those used by its competitors. To create this massive database, the co-founders, both former AI researchers at Meta, combined various data collection techniques they had developed and tested over years of research.
One method was to hire human contractors to remotely control the robots and collect data on their behavior. Another was to have the robots perform random tasks, record the results, and let them learn by trial and error. The AI models were also trained on millions of public videos.
As a doctoral student at the University of California, Berkeley, Pathak developed a way to instill “artificial curiosity” in robots, rewarding them for producing outcomes when the system couldn't predict the outcome of its actions. “The less an agent can predict the impact of its actions, the more curious it becomes to explore,” he explained. The technique motivated the AI to navigate more scenarios and gather more data.
His research on curiosity-driven learning was published in 2017 and has been cited more than 4,000 times, he said. Pathak has also devised ways for robots to use written information from large-scale language models like GPT (for example, how to open a milk can) and translate it into actions.
“In 2022, we figured out how to bring these things together into one coherent system,” Pathak said.. “It's the concept of learning from video, learning from curiosity, learning from real data, but combining it with knowledge from simulation.”
Trained on 1,000 times more data than its competitors, Skild AI's underlying models can be built into a wide range of robots, enabling them to do things like climb steep slopes, walk over objects that block their path, and identify and pick up items.
Skilled AI
SkilledAI faces stiff competition from a string of robotics companies that have garnered billions of dollars in venture funding thanks to the AI boom. Industry giant OpenAI recently revived its robotics team to supply models to the robotics companies. Forbes Among the first to be reported were Figure AI, a humanoid robotics company led by billionaire CEO Brett Adcock, and Covariant, an OpenAI spinoff that's building ChatGPT for robots and has raised more than $200 million to do so.
Co-founder Gupta claims that Skild AI's access to large amounts of data is what differentiates it from others in the field, but he declined to disclose the exact amount of data its models are trained on.
Ken Goldberg, a professor of robotics and automation at the University of California, Berkeley, agrees that data is key to scaling robotics, but robots need certain kinds of data that aren't widely available on the internet, and using data collected from simulations doesn't necessarily translate to the real world.
“What robotics is looking at right now is being able to do something similar to large-scale language models or large-scale visual language models, both of which have access to internet-scale data and have billions of examples,” he said. This is no easy task for robotics, but Skild AI aims to address the problem by combining all of its data collection techniques with more information coming from simulations.
Pathak and Gupta envision their company's future being similar to OpenAI, where Skilled's foundational model can be tweaked to build a variety of use cases and products. “That's where we're aiming to revolutionize the robotics industry,” Gupta says, adding that they ultimately want to achieve artificial general intelligence for robots—virtual AI systems that match or exceed human capabilities—but that humans can interact with in the real world.
“The world of robotics has its GPT-3 moment,” said Stephanie Zhang, a partner at Sequoia Capital and an existing investor in SkilledAI. “It will ignite a revolution that brings the same advancements we've seen in the world of digital intelligence to the physical world.”
More from Forbes
