The satellite used a vision language model to autonomously locate targets in orbit, proving that onboard AI can prioritize images before downlinking.
For the first time, an Earth observation satellite has detected the object it is looking for on its own, without the involvement of human analysts on the ground. The achievement, documented in April, is the first time the use of visual language models (VLMs) has been documented in orbit and shows how artificial intelligence can fundamentally change the capabilities of space sensors and their value.
Typically, satellites transmit ground data to analysts, and machine learning algorithms or the human eye are used to interpret the situation. But on Loft Orbital’s Yam-9 spacecraft, a software package from NASA’s Jet Propulsion Laboratory responded to natural language queries to identify areas of interest.
Google DeepMind’s Gemma 3 is the very visual language model used in the demonstration. It’s designed for edge devices, so it can run on limited hardware outside of your data center. VLM combines the contextual understanding and image analysis capabilities of large-scale language models. Researchers asked the model to classify sensor data where nature and human development meet, or to detect infrastructure around rail hubs.
Demonstrations are important for two reasons. First, in the short term, space sensors could become even more useful by performing data preprocessing in orbit and reducing the stream of raw data that is analyzed on Earth. The second is a proof of concept for the possibility of launching more scalable AI infrastructure into space.
“With VLM, you can build logic, say, to monitor this perimeter and report if something looks suspicious,” he says, “and you can talk to the satellite.”
– Paul Lasserre
Technical details and next steps
Loft develops its technology as a platform for external clients. This business model is closer to infrastructure-as-a-service than traditional satellite manufacturing. The recent contract plans for EarthDaily to create, launch and operate six new satellites that will analyze and sell the data it collects.
Yam-9 will be launched in fall 2025 as the flagship product of Loft’s AI policy and is powered by the Nvidia Jetson Orin AGX graphics processor, one of the leading chips used in space computing.
Juan Delfa Victoria, a technology leader in NASA JPL’s AI group, led the development of NAVI-Orbital, a software package that is essentially the “chassis” of the Gemma 3 VLM. Although Gemma 3 is an off-the-shelf solution, the developers have simplified the packaging and reduced the number of libraries and memory requirements.
This is the first recorded use of VLM in orbit, but others are expected to follow. Planet Labs uses satellites powered by Jetson Orin processors. These are currently being used for simpler object detection tasks, but research is underway for other AI applications, including VLM.
Kepler Communications, which operates the largest fleet of GPUs in space, declined to say whether it had deployed VLM due to NDAs with partners, but noted that there have been several undisclosed use cases of its computing environment since the spacecraft’s launch.
“Now that we have proof of concept, it’s a milestone that we’re really moving forward with,” Lasserre said. “The goal is to build a constellation of satellites that can cover any point on Earth in real time, which officials say will require 50 to 100 Yam-9 satellites. Loft currently operates 12 spacecraft in orbit.”
The lessons learned from deploying these small models into orbit will help explain how companies will deploy large-scale computing infrastructure in space in the future, especially for routine but critically important power management and memory management problems.
It could also open the door to new scientific tools. The idea for NAVI-Space came from the mind of JPL researcher Taran Cyriak John, who was thinking about a digital assistant for astronauts exploring the Moon and Mars.
“We believe that astronauts need assistance, similar to video games and movies where artificial intelligence is an interactive helper,” Delpha Victoria said.
“And it shouldn’t be called HAL 9000,” they added, in the context of caution in naming general-purpose systems.
Future prospects suggest that VLM technology has the potential not only to fundamentally change how data from space platforms is processed, but also to prepare new approaches to science and research in orbit and beyond.
The adjusted pace of development indicates the need for greater focus on practical, energy-efficient computing solutions in space, where resources are as valuable as data from the stars.
Looking to the future, the first wave of VLM use in orbit could spur the creation of more autonomous space systems that can uniquely understand the context of images and make decisions without constant control on the ground.
