NVIDIA RTX technology is deeply integrated into Unreal Engine 5 through the NVIDIA RTX branch of Unreal Engine and the NVIDIA DLSS Unreal Engine plugin. This gives developers direct access to advanced rendering, frame generation, and ray-traced lighting. NVIDIA is expanding this integration with new tools for building on-device AI characters and gameplay, as announced at Unreal Fest 2026.
This post is about:
Build on-device AI companions more easily with NVIDIA ACE
AI is opening up possibilities for next-generation NPCs that offer experiences never before possible. For example, KRAFTON recently introduced its AI teammate, Ally. PUBG: Battlegrounds Use natural voice interactions to understand player intent, interpret gameplay, and respond dynamically in real-time. Rather than following a strict script, Allies collaborate, adapt to combat and looting, and communicate naturally. It also features GeForce RTX to run AI completely on-device. PUBG Ally will be available in open beta soon.
Building on-device AI NPCs requires not only choosing the right AI models, but also integrating them into a seamless, high-performance pipeline. To deliver a workable experience, developers must overcome hurdles such as minimizing conversation latency for instantaneous responses, synchronizing global dynamic game state, and preventing NPCs from getting stuck in infinite loops.
NVIDIA ACE Game Agent SDK Beta and Unreal Engine 5 plugins make it easier for game developers and ISVs to build and deploy responsive, on-device AI companions.
What is the NVIDIA ACE Game Agent SDK?
NVIDIA ACE Game Agent SDK is a lightweight C/C++ agent framework designed for seamless native in-game integration. It is an open source, fully customizable solution that is optimized for small models and hardware accelerated with NVIDIA RTX.
Developers interact with the SDK through three core API categories:
- Agent API: A stateful API that owns the chat history and autonomously drives tool-assisted multi-step inference with minimal application code.
- Chat API: Stateless API that gives developers direct control over inference
- Lag API: API that retrieves semantic, lexical, and hybrid knowledge from developer-built databases and enables model responses based on up-to-date knowledge
Together, these tools allow NPCs to dynamically perceive, reason, and act within the game world based on real-time player input. This tool is used and battle-tested in real games, including an experimental in-game AI advisor. Total War: Pharaoh. Depicted as an ancient Egyptian pharaoh, this advisor provides situational advice for a playtest program starting in 2026, including recommendations for court actions and building construction to quell rebellions. It uses a search augmentation generation (RAG) architecture that queries over 1,200 interlinked game data tables to answer questions about player strategy in real time.
Download the beta version of the NVIDIA ACE Game Agent SDK to start building the future of gaming.
What does the new suite of NVIDIA ACE Unreal Engine 5 plugins include?
A new suite of NVIDIA ACE plugins is now available for Unreal Engine 5 developers and technical artists. These local runtime AI models support both out-of-the-box Blueprint and C++ integration and cover three pillars:
- Automatic speech recognition (ASR): Includes a ready-to-use English model, nemo-conformer-ctc-120m, with download options for 7 additional languages, sample levels, and blueprint samples for quick setup.
- Small Language Model (SLM): It provides local GGUF support for low-latency text generation and function call functionality, is optimized for instant character interaction and decision-making, and also includes a ready-to-use Qwen 3.5 4B model.
- Text-to-speech (TTS): Includes a high-quality Chatterbox Turbo 350M TTS model, sample voices, sample levels, and exclusive plug-in content to instantly bring your characters to life.
These plugins allow creators to implement live transcription, dynamic character interaction, voice commands, and contextual action responses directly within their game logic.
Unlike cloud-based services that suffer from high latency and unpredictable operational costs, these plugins offer local RTX-optimized workflows bundled with ready-to-use models. These provide an instant, end-to-end AI NPC pipeline directly into your Unreal Engine 5 project.
Get started by downloading the NVIDIA ACE Unreal Engine 5 plugin, and join the NVIDIA Developer Discord community for support and discussion.
To learn more, register for NVIDIA’s live Level Up webinar “Building AI-Powered NPCs in Unreal Engine 5.7” on June 30th. The webinar will provide details about these plugins and give you the opportunity to ask questions.
What is the new NVIDIA DLSS 4.5 Unreal Engine plugin?
The new DLSS 4.5 Unreal Engine plugin adds support for DLSS 4.5 dynamic multi-frame generation, multi-frame generation 6X mode, and a super-resolution second resolution transformer.
This plugin provides direct access to the latest DLSS super resolution and frame generation updates. Built on Streamline, this plugin provides a consistent integration path across DLSS functionality, allowing developers to selectively adopt features such as ray reconfiguration and dynamic multi-frame generation. Updated APIs, documentation, and sample code reduce integration time and make it easier to deploy DLSS in both new and existing projects.
Learn more at Unreal Fest 2026
Join us at NVIDIA’s Unreal Fest 2026 to learn how NVIDIA ACE and Unreal Engine are redefining the future of interactive character performance.
The June 17th session, “Ready, Set, Action: Why Your Next NPC Should Be Their Own Method Actor,” describes a new workflow for building autonomous game characters. role NPC’s Performers. Learn how to create AI personalities that are deeply responsive and always unique. Using synthetic training data generated by a large-scale supervised model, you can distill character behavior into lightweight, deployable, LoRA-powered performance that runs locally on NVIDIA GeForce RTX hardware.
This session explains how game developers can easily create interactive MetaHuman NPCs in Unreal Engine 5.7 that run completely on-device using NVIDIA ACE.
- ACE UE voice plugins: Riva Parakeet ASR for real-time speech recognition, Chatterbox TTS for expressive speech synthesis
- Facial animation and performance delivery by Audio2Face
Modern workflows allow NPCs to impersonate and dynamically perform as a character during gameplay, then break the character and work directly with developers to adjust motivation, generate backstory material, and help build future training data.
Authentic first person perspective chalk war Games with AI-powered features
The studio in the back chalk war SOKRISPYMEDIA, a series with 350 million views, is building an authentic first-person action game in the franchise, whose distinguishing features are based on local AI. Players hand-draw their weapon of choice and integrate it into the game in seconds using a model trained to handle how each weapon is held, fired, recoil, and reloaded.
To train a bespoke matching model without paying for expensive cloud computing, the team ran it on a Puget Systems workstation built around dual NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition GPUs. Real-time rendering veterans are moving from 2D content to playable media using hardware tailored for AI training. To see the game in action at Unreal Fest, visit the Puget Systems booth.
Rapid and controllable human motion now available in Unreal Engine
NVIDIA has released NVIDIA Kimodo, an open source project for rapid and controllable human movement. Describe motion in easy-to-understand language (“a person walks happily forward and jumps into the air”), lock in specific bones and keyframes throughout the body for precise art direction, and generate complex human motion in seconds.
Building on this project, Animotive Kimodo is a plugin that brings Kimodo directly into Unreal Engine, allowing animators to adjust, layer, and handkey AI-generated motion on their rigs without leaving the editor. AI Motion is not a locked output to replace crafting, but a starting point.
The Unreal Engine plugin ships today, with Blender and Maya integration planned. Available self-hosted or through Animotive Cloud. An upcoming update will allow AAA studios to securely upload their own animation datasets and train Animotive Kimodo to follow their own in-house style. Learn more about the launch of Animotive Kimodo here.
Try using NVIDIA ACE and DLSS 4.5 with Unreal Engine 5
At Unreal Fest 2026, NVIDIA will announce further features that enable on-device AI characters and gameplay through NVIDIA ACE, including the ACE Game Agent SDK, new Unreal Engine 5 plugins, and DLSS 4.5.
Mark your calendars to join NVIDIA AI research leaders Sanja Fidler, Ming-Yu Liu, and David Luebke at SIGGRAPH 2026 on July 20th to explore the latest breakthroughs shaping the future of computer graphics and simulation.
Resources for game developers
Check out the complete list of game developer resources and stay up to date with the latest NVIDIA game development news.
