Editor’s note: This post is part of To Omniversea series focused on how developers, 3D practitioners, and businesses can take advantage of the latest advances to transform their workflows. open usd and NVIDIA Omniverse.
Cities around the world are facing unprecedented challenges due to rapidly growing urban populations and the strain on infrastructure to accommodate them.
Operational challenges such as traffic congestion and emergency service coordination are exacerbated by fragmented data pipelines, siled local government processes, and disparate systems. Technical barriers prevent cities from accessing comprehensive, real-time insights needed for effective decision-making and city management.
Major cities and technology partners are deploying NVIDIA Blueprint for Smart City AI, a reference application that provides a complete software stack to build, test, and operate AI agents in simulation-ready (SimReady) digital twins.
OpenUSD is an open and extensible framework that connects each stage of this physics AI workflow. OpenUSD-enabled digital twins serve as SimReady environments where cities can simulate “what if” scenarios and generate physically accurate sensor data.
Blueprints power a three-step workflow. 1) Simulate and generate synthetic data using the NVIDIA Cosmos platform and NVIDIA Omniverse libraries, 2) Train and fine-tune vision AI models, and 3) Deploy real-time video analytics AI agents using the NVIDIA Metropolis platform and NVIDIA Blueprints for Video Search and Summarization (VSS). This will enable cities to move from reactive to proactive operations.
Based on these simulations, cities can deploy operational platforms with integrated weather data, traffic sensors, and emergency response systems to support rapid testing of rare scenarios, real-time monitoring, urban infrastructure planning, and urban system optimization.
From Kaohsiung, Taiwan, where street-level AI reduced incident response time by 80%, to Raleigh, North Carolina, which achieved 95% vehicle detection accuracy, to France’s rail network, which optimized energy consumption by 20%, cities around the world are using digital twins and AI agents to transform city operations at scale.
Smart city practice
SNCF Gares&Connexions’ Akila uses digital twins to improve rail operations
Akila’s digital twin application helps French rail operator SNCF Gares&Connexions optimize its network of nearly 14,000 trains daily using live scenario planning for solar heating, airflow and crowd movement. OpenUSD-enabled digital twins reduce energy consumption by 20%, enable 100% on-time preventive maintenance, and reduce downtime and response times by 50%.
Linker Vision leverages physical AI to deliver street-level intelligence
Linker Vision’s physical AI system recognizes infrastructure events in Kaohsiung City, such as broken street lights and fallen trees, eliminating manual city inspections and enabling faster emergency response. To extend street-level intelligence to more cities, Linker Vision uses the Omniverse library for simulation, Cosmos Reason for world understanding, and VSS Blueprints for deployment with OpenUSD.
Esri and Microsoft bring comprehensive city intelligence to the City of Raleigh
The City of Raleigh uses the NVIDIA DeepStream Software Development Kit to achieve 95% vehicle detection accuracy and enhance engineers’ traffic analysis workflows. This data powers Raleigh’s digital twin, enabled by Esri’s ArcGIS geospatial platform, to support visualization and analysis for critical infrastructure planning and management. Integrating this computer vision pipeline with a vision AI agent powered by NVIDIA VSS blueprints provides comprehensive real-time visibility and insights in ArcGIS on Azure Cloud.
Video Review Automation with Milestone Systems’ VLM
Milestone Systems will soon launch Hafnia VLM, which includes a VLM plugin and VLM-as-a-service for video management software XProtect. Fine-tuned based on over 75,000 hours of video data, Hafnia VLM can reduce operator alarm fatigue by up to 30% by automating video reviews and filtering false alarms. It was developed using NVIDIA Cosmos Reason VLM and Metropolis. The Hafnia VLM plugin for XProtect gives XProtect operators and users easy access to generative AI.
K2K analyzes Italian video streams
K2K’s platform uses NVIDIA Cosmos Reason and VSS Blueprints to analyze more than 1,000 video streams in Palermo, Italy, processing 7 billion events annually and automatically notifying city officials through natural language queries and video events when critical conditions are extracted and analyzed.
Watch this on-demand NVIDIA GTC session, “Leadership Strategies to Transform Public Services,” to learn more about how cities are transforming with simulation, vision AI, and digital twins.
Get started with smart city AI
To learn more about OpenUSD and computer vision workflows, check out the following resources:
Subscribe to stay up to date NVIDIA Omniverse Newsjoin the Omniverse community And following the Omniverse discord, Instagram, linkedin, thread, × and YouTube.
