Video Analytics Age AI Agent is here.
Video is one of the crucial features of the modern digital landscape, accounting for over 50% of all global data traffic. It is one of the largest and most ubiquitous data sources in the world as it is dominant in the media and is increasingly important to businesses across the industry. However, less than 1% have been analyzed for insight.
Almost half of global GDP comes from the physical industry, which spans energy across automobiles and electronics. With concerns about labor shortages, efforts to make ends meet during manufacturing, and growing demand for automation, video analytics AI agents can play a more important role than ever before and help fill the physical and digital world.
To accelerate the development of these agents, Nvidia is today creating a video search and summary (VSS) AI blueprint powered by the Nvidia Metropolis platform.
A wave of Vision AI agents and productivity assistants with Vision Language Models (VLM) is now online. Combining the skills of powerful computer vision models with the highly intelligent leading language models (LLMS), these video analytics AI agents allow companies to easily view, search and summarise huge amounts of videos. By analyzing videos in real time and reviewing terabytes of recorded videos, video analytics AI agents unlock unprecedented value and opportunity within a critical industry range.
Manufacturers and warehouses use AI agents to help increase workers' safety and productivity. For example, agents can help place forklifts and workers for optimal efficiency. SmartCity has deployed AI agents for video analytics to reduce traffic congestion and increase safety, and will continue to use.
https://www.youtube.com/watch?v=pw8tl_bjnwa
Blueprint AI Agent for Creating a Diverse Fleet of Video Analytics
The VSS Blueprint is built on the Nvidia Metropolis platform and is boosted by VLMS and LLMs such as Nvidia Vila, Nvidia Llama Nemotron, Nvidia Nemo Retriever Microservices, and Redirival-Augmented Generation (RAG).
VSS BluePrint incorporates the NVIDIA AI Enterprise Software Platform, including NVIDIA NIM Microservices for VLMS and an advanced AI framework for RAG. VSS Blueprint allows users to summarize videos 100 times faster than they see in real time. For example, you can put an hour of video into text within an hour.
VSS Blueprint offers many powerful features designed to provide robust video understanding, performance and scalability.
This release introduces enhanced hardware support, including the ability to deploy small workloads on a single NVIDIA A100 or H100 GPU, providing greater flexibility in resource allocation. The blueprints can also be deployed on the NVIDIA RTX 6000 Pro and the Edge of the NVIDIA DGX Spark computing platform.
VSS Blueprints can process hundreds of live video streams or burst clips simultaneously. It provides audio transcription in addition to visual understanding. Converting speeches into text adds depth of context to audio-critical scenarios such as training videos, keynotes, and team meetings.
Industry leaders use video analytics to drive business value
Everyone from world-leading manufacturers to smart cities and sports leagues uses VSS Blueprints to develop AI agents to optimize operations.
Pegatron, a leading electronics manufacturer, uses VSS blueprints to investigate operational procedures and train employees with best practices. The company is also integrating blueprints into the Pegaai platform so that organizations can build AI agents to transform their manufacturing processes.
These agents ingest and analyze large volumes of videos, enabling advanced features such as automatic surveillance, anomaly detection, video search, and incident reporting. Pegatron's visual analysis agent allows you to understand the operating procedures for printed circuit board assembly and determine whether the actions are correct or incorrect. To date, agents have reduced Pegatron labor costs by 7% and defect rates by 67%.
Additional major Taiwanese semiconductor and electronics manufacturers are building AI agents and digital twins to optimize their planning and operational applications.
Taiwan's Kaohsiung City uses a unified Smart City Vision AI application developed by partner Linker Vision to improve incident response times. Previously, urban sectors such as waste management, transportation and emergency response were separated by siloed infrastructure. This resulted in slow response times due to lack of access to important information.
Linker Vision's AI-powered application with VSS Blueprint combines real-time video analytics and generated AI to not only detect visual elements, but also understand and narrate complex urban events such as floods and traffic accidents.
Linker Vision currently provides timely insights to 12 urban sectors, and by 2026 it is on track along over 50,000 expansions from 30,000 urban cameras. These insights improve situational awareness and data-driven decision-making across urban services, reducing incident response times by up to 80%.
https://www.youtube.com/watch?v=1ocukybmpw4
The National Hockey League streamlined and accelerated its Vision AI workflow using a vast insight engine using VSS Blueprints. Manages a large amount of game footage.
With a vast amount of insight, the NHL is positioned to search for petabytes of videos in sub-seconds, allowing for highlights and searches near in-game moments. AI-driven agent workflows further enhance content creation by automatically clipping, tagging and assemble video content for easy access and use.
In the future, the league may use real-time AI inference to enable tailored insights, such as player statistics, strategic analysis, and fantasy recommendations, which were dynamically generated during live games. This end-to-end automation can change the way media is created, curated and distributed, setting new standards for AI-driven sports content production.
https://www.youtube.com/watch?v=w1igbdpfpde
Siemens uses industrial co-pilots to help factory floor workers with equipment maintenance, error handling and performance optimization. This generative AI-powered assistant uses information about operational and document data to provide real-time answers to device errors.
The co-pilot was built with a fusion of VSS components such as VLMS, LLMS, and Nvidia Nemo microservices. Industry co-pilots have led to rapid decision-making and reduced downtime for the machine. Siemens reports a 30% increase in productivity, potentially reaching 50%.
Supported by a growing partner ecosystem that creates sophisticated AI agents
Nvidia Partners uses VSS Blueprints to accelerate the creation of agent AI video analytics capabilities for workflows, reducing development time from months to weeks.
SuperB AI, a leader in intelligent video analytics, has established a sophisticated airport operation project at Incheon Airport to reduce passenger waiting times in a few weeks. In Malaysia, solution provider Itmax has built an advanced visual AI agent with Kuala Lumpur city VSS blueprints to improve overall city management and reduce incident response times.
In the advertising sector, Pyler integrated VSS Blueprint into brand safety (AID) and AD targeting (AIM) solutions in just a few weeks. Using AID and AIM, Samsung Electronics has increased the effectiveness of its advertising with high value ad placement that is consistent with brand and product. BYD has increased its rates four times via ad clicks by targeting positive content that is contextually relevant, and HANA Financial Group surpasses multiple brand campaign targets.
Fingermark is an application provider for EyeCue, a real-time computer vision platform used by quick service restaurants. Fingermark adds VSS blueprints to EyeCue to turn video footage into clear and actionable insights into drive-through latency, service bottlenecks and staff-related incidents.
Try VSS Blueprints at build.nvidia.com. Read this technical blog for more information.
Please take a look Computex Keynote From Nvidia Founder and CEO Jensen Huang and Nvidia GTC TAIPEI 2025 session.
