Toyota reveals grand AI vision for automotive and beyond

Diving overview:

• At a recent event marking the first open house of Toyota Motor Corporation’s Woven City Mobility Experiment Station near Japan’s Mount Fuji, the automaker unveiled its AI Vision Engine, a large-scale AI foundational vision language model that integrates visual, behavioral, and environmental data.
• Toyota claims its AI vision engine can identify patterns, detect potential risks, and enable coordinated action across connected systems to improve safety. It will be a core component of the automaker’s next-generation Anzen driver assistance system and Arene software platform for millions of software-defined vehicles produced in a wide range of configurations.
• Toyota is also considering commercializing AI vision engines for products other than automobiles. It employs AI to understand and analyze video data and has been trained from the ground up, and Toyota says this could also be applied to retail environments, airports and offices.

Dive Insight:

At the Woven City event, which WardsAuto also attended, Toyota further touted the capabilities and versatility of its AI vision engine, which was built entirely in-house by the automaker’s software subsidiary, Woven by Toyota.

The automaker said in a release that it is one of the world’s leading VLMs as ranked by MVBench Leaderboard. Toyota’s software division had automotive and mobility in mind when it developed its AI vision engine, but it’s also looking at applications far beyond that.

The AI ​​vision engine can understand the behavior of people, objects, and mobility through camera footage, Woven said. However, facial recognition is not used. What sets it apart from other VLMs is that it can quickly summarize what’s happening from the camera’s video, report what happened, and help predict what will happen, officials explained.

“For us, self-driving is really a matter of learning,” said Dushyant Wadibkar, global head of AD/ADAS at Toyota Woven by Toyota. “Rather than having an algorithm specifically programmed for a particular scenario, it’s about a system that can learn and continuously process the subtle aspects of driving.”

To solve learning problems, Woven by Toyota is building two core products. A machine learning stack designed to learn from drivers, vehicle data, and road conditions. So is the entire ecosystem that powers the stack. These make up what Wadivukal calls an “active learning loop,” which allows Toyota to extract data from millions of vehicles to train machine learning models, validate and validate the data, and deploy it to customers.

SDVs that receive regular over-the-air updates define a direct relationship with end consumers. So this ecosystem, where brand image is at stake, is something Toyota wants to oversee every aspect of, including the execution platform, which needs to be compliant with regulators.

Toyota plans to build the platform in stages over several years toward the realization of a complete SDV. As Toyota CTO John Absmeyer explained, the automaker intentionally started with multimedia and ADAS, which he said are the two most complex elements.

“But it’s not just about learning for ADAS,” Absmeyer said. “We’re talking about an active learning loop for all purposes inside the car: learning how users use it in a specific region, in a specific environment, and under specific regulatory and certification differences.”

With more than 100 million Toyota vehicles on the road worldwide, there are dozens of markets where Toyota differentiates its product lines and model variations differently to provide vehicles with the right features and pricing. “This poses a huge challenge when trying to deliver software to millions of units that must be improved over time,” said Jean-François Campeau, head of Arene and vice president of Woven by Toyota.

To overcome build variations and potential incompatibilities, Arene is moving into what Absmeyer calls the “postdomain era,” using software tools that don’t care about the hardware and can essentially test and develop software independently of the hardware.

The machine learning stack is designed to be “universal,” emphasizes Wadibukal, who has previously focused on self-driving products at Bosch, Cruise, and Torque Robotics, so it can be deployed in any car. “We’re not designing a specific application for a specific product, but we’re designing an application system that can work with a large number of vehicles that Toyota and Lexus own,” he said.

Wadivkar pointed out that as part of a vehicle safety system, there is no need to have separate applications for a five-sensor environment and one for a six-sensor layout, and there is no need to redesign the system if a different LIDAR supplier is used.

Absmeyer said Toyota plans to roll out Arene globally and across its product lineup “over the next few years,” bringing the system into “everything from ultra-low-end cars to ultra-ultra luxury cars and every region of the world.”

arene debuts for the 2026 Toyota RAV4’s infotainment and safety systems. The Lexus ES is expected to arrive later this year with a similar Arene, and Toyota says its next generation EV will be built on it.

Vadifkar said the active learning loop will be fully automated, but Toyota currently always has a developer on hand to oversee the entire process. Toyota, which is actively testing Arene-based systems in California, Michigan and Japan, says Woven City allows it to develop new ideas and is willing to engage the city’s “weavers.” They’re probably the ones who are there with their families and make things with participating businesses.

Toyota also uses a team of professional master drivers, usually responsible for product development, to help identify what “good driving” actually means, but Wadifkar acknowledges that this is difficult and situational. “What constitutes good driving is a very difficult question, but if we can get this opinion from experienced and elite drivers, we can manage and use the data very effectively,” he said.