Skild AI brain allows robots to learn daily tasks by watching videos

Thanks to Skild’s new AI models, robots are learning from videos to perform tasks.

The system allows robots to learn skills by observing videos of humans and performing tasks such as opening doors, watering plants, assembling boxes, and even cooking with less than an hour of robot-specific training data.

The company says its Skild Brain is resistant to external disturbances, can be generalized to new homes without retraining, and works with a variety of robot types, from wheeled humanoids to two-handed setups.

In testing, multiple AirPod cases can be accurately assembled in succession. Skild Brain attributes its success to in-context learning and plans to further expand its capabilities.

shared brain robotics

In the race to develop truly general-purpose robotics, the industry has hit a familiar roadblock: the data bottleneck.

While large-scale language models have thrived on vast datasets collected from the Internet, robotics has lacked a comparable “common crawl.” Traditional approaches rely heavily on remote control, where a human manually guides the robot to perform precise motor torque sequences. But robotics companies argue that this method alone cannot scale to the basic model level.

Remote operations face two important limitations. First, diversity: data is limited to controlled labs or specific deployments and lacks the chaotic variability of real-world environments. Second, scale: Even with a global workforce, it is virtually impossible to collect trillions of action sequences in real time.

The company says that unlike traditional AI models designed for specific types of robots, its underlying model is full-vehicle, meaning it can control any robot, including humanoids, quadrupeds, tabletop arms, and mobile manipulators, without prior knowledge of its exact shape.

The company’s Skild Brain enables robots to perform everyday tasks such as cleaning, loading the dishwasher, and cooking, as well as physically demanding tasks such as navigating slippery or uneven terrain.

adaptive robot intelligence

Founded in 2023 by experts in self-supervised adaptive robotics, Skild AI develops scalable foundational models that act as a shared brain across various robot embodiments.

The company is addressing a major challenge in robotics: the absence of an “internet of robotics.” Unlike language and video models, robotics has historically relied on limited datasets collected through remote control or individual deployment. To overcome this, the company pre-trains Skild Brain using human videos from the internet and physics-based simulations, allowing it to acquire skills without extensive robot-specific data.

Skild Brain can operate in all directions and across multiple robot forms, including humanoids, quadrupeds, mobile manipulators, and tabletop arms. Adapt to unpredictable situations like missing limbs, jammed wheels, increased payload, or completely new bodies in real time without retraining or tweaking.

He said the model emphasizes adaptation over memory, creating a continuous learning cycle that improves performance with each deployment, regardless of the robot’s hardware or the task at hand. Robot report.

This approach is inspired by human learning, where skills are acquired through observation and intention rather than precise force or movement. Video, from first-person headcams to millions of educational clips online, provides a vast and previously untapped dataset for robotics. To fill the implementation gap, Skild Brain maps human movements onto different robot shapes and fills in missing tactile and force data.

By learning from video demonstrations supplemented with minimal robot data, the model fundamentally overcomes the robot data bottleneck and enables scalable general-purpose task learning, according to Skild.