NVIDIA Brings AI Assistants to Life with GeForce RTX AI PCs

COMPUTEX—NVIDIA today announced new NVIDIA™ technology allows AI assistants and digital humans to What's New in NVIDIA GeForce RTX 5000™ AI Laptop.

NVIDIA has announced “Project G-Assist,” an RTX-powered AI assistant tech demo that provides contextual help for PC games and apps. ARK: Survival Ascended From Studio Wildcard. NVIDIA also NVIDIA ACE Digital human platform.

These technologies include: NVIDIA RTX AI Toolkitis a new suite of tools and software development kits that help developers optimize and deploy large-scale generative AI models on Windows PCs. They join NVIDIA's full-stack RTX AI innovations that accelerate more than 500 PC applications and games, and more than 200 laptop designs from manufacturers.

Additionally, the newly announced RTX AI PC laptops from ASUS and MSI feature up to GeForce RTX 4070 GPUs and power-efficient systems on a chip with Windows 11 AI PC capabilities. These Windows 11 AI PCs will receive a free update to the Copilot+ PC experience when it becomes available.

“NVIDIA ushered in the era of AI PCs in 2018 with the release of RTX Tensor Core GPUs and NVIDIA DLSS,” said Jason Paul, vice president of Consumer AI at NVIDIA. “Now, with Project G-Assist and NVIDIA ACE, we're bringing next-generation AI-powered experiences to more than 100 million RTX AI PC users.”

GeForce AI Assistant Project G-Assist
AI assistants are poised to transform games and in-app experiences, from providing game strategies and analyzing multiplayer replays to assisting with complex creative workflows. Project G-Assist is a glimpse into this future.

PC games offer vast worlds to explore and complex mechanics to master, making them a challenging and time-consuming task for even the most dedicated gamers. Project G-Assist aims to use generative AI to put gaming knowledge at the player's fingertips.

Project G-Assist takes voice or text input from the player and contextual information from the game screen, then passes that data to AI vision models that enhance the contextual awareness and app-specific understanding of Large Language Models (LLMs) linked to a game knowledge database to generate customized responses delivered as text or speech.

NVIDIA partnered with Studio Wildcard to demonstrate the technology. ARK: Survival AscendedProject G-Assist helps answer questions about creatures, items, lore, objectives, difficult bosses, etc. Because Project G-Assist is context aware, it customizes responses to fit the player's game session.

Additionally, Project G-Assist can configure a player's gaming system for optimal performance and efficiency, providing insight into performance metrics, optimizing graphics settings based on the user's hardware, applying safe overclocking, and intelligently reducing power consumption while maintaining performance targets.

First ACE PC NIM debuts

NVIDIA ACE Technology ACE for augmenting digital humans comes to RTX AI PCs and workstations powered by NVIDIA NIM, an inference microservice that helps developers reduce deployment time from weeks to minutes. The ACE NIM microservice provides high-quality inference running locally on device for natural language understanding, speech synthesis, facial animation and more.

COMPUTEX will feature the gaming debut of the NVIDIA ACE NIM on the PC. Covert Protocol Technology DemoDeveloped in collaboration with Inworld AI. NVIDIA Audio 2 Face™ and NVIDIA Riva Automatic speech recognition that runs locally on the device.

Windows Copilot runtime adds GPU acceleration to local PC SLM
Microsoft and NVIDIA are collaborating to help developers bring new generative AI capabilities to their native Windows and web apps. The collaboration will give application developers easy application programming interface (API) access to GPU-accelerated small language models (SLMs), Search Extension Generation A feature that runs on the device as part of the Windows Copilot Runtime (RAG).

SLM offers tremendous possibilities for Windows developers, including content summarization, content generation, and task automation. The RAG feature extends SLM by enabling AI models to access domain-specific information that is not adequately represented in the base model. The RAG API allows developers to leverage application-specific data sources and tailor SLM behavior and capabilities to the needs of their application.

These AI capabilities are accelerated by NVIDIA RTX GPUs and AI accelerators from other hardware vendors, delivering fast and responsive AI experiences to end users across the Windows ecosystem.

The API will be released as a developer preview later this year.

RTX AI Toolkit Delivers 4x Faster, 3x Smaller Models
The AI ​​ecosystem has built hundreds of thousands of open-source models that app developers can leverage, but most models are pre-trained for general purposes and built to run in data centers.

To enable developers to build application-specific AI models that run on PCs, NVIDIA RTX AI Toolkit — RTX AI A suite of tools and SDKs for customizing, optimizing and deploying models on PC. The RTX AI Toolkit will be released later this month to make it accessible to a wider range of developers.

Developers can use the open source QLoRa tools to customize the pre-trained models. NVIDIA Tensor RTQuantize your model using the NVIDIA TensorRT™ model optimizer to reduce RAM consumption by up to 3x. The NVIDIA TensorRT Cloud then optimizes your model for best performance across the entire RTX GPU lineup, resulting in up to 4x performance improvement compared to pre-trained models.

new NVIDIA AI Inference Manager The SDK, currently available in early access, simplifies the deployment of ACE on PC by pre-configuring the PC with the necessary AI models, engines and dependencies, and seamlessly orchestrating AI inference across the PC and the cloud.

Software partners including Adobe, Blackmagic Design and Topaz are integrating components of the RTX AI Toolkit into popular creative apps to accelerate AI performance on RTX PCs.

“Adobe and NVIDIA continue to collaborate to deliver breakthrough customer experiences across all creative workflows, from video to imaging, design, 3D and more,” said Deepa Subramaniam, vice president of Creative Cloud product marketing, Adobe. “TensorRT 10.0 on RTX PCs provides creators, designers and developers with unprecedented performance and AI-accelerated capabilities, unlocking new creative possibilities for content creation in industry-leading creative tools like Photoshop.”

Components of the RTX AI Toolkit, including TensorRT-LLM, are integrated into popular developer frameworks and applications for generative AI, including Automatic1111, ComfyUI, Jan.AI, LangChain, LlamaIndex, Oobabooga and Sanctum.AI.

AI for Content Creation
NVIDIA is also integrating RTX AI acceleration into apps for creators, modders, and video enthusiasts.

Last year, NVIDIA introduced RTX acceleration with TensorRT for one of the most popular Stable Diffusion user interfaces, Automatic1111. Starting this week, RTX will also accelerate the popular ComfyUI, delivering up to 60% better performance than the currently shipping version and 7x better performance than the MacBook Pro M3 Max.

NVIDIA RTX Remix is a modding platform for remastering classic DirectX 8 and DirectX 9 games with full ray tracing, NVIDIA DLSS 3.5, and physically accurate materials. RTX Remix includes a runtime renderer and the RTX Remix Toolkit app for easy modding of game assets and materials.

Last year, NVIDIA open-sourced the RTX Remix Runtime, allowing modders to Extend game compatibility It has advanced rendering capabilities.

Since the RTX Remix Toolkit was released earlier this year, 20,000 modders have used it to Classic GamesAs a result, over 100 RTX remasters are in development. RTX Remix Showcase Discord.

This month, NVIDIA is open-sourcing the RTX Remix Toolkit, streamlining how modders replace assets and relight scenes, expanding support for more file formats in RTX Remix's asset ingestor, and enabling RTX Remix's AI texturing tools to be enhanced with new models.

Additionally, NVIDIA is making the RTX Remix Toolkit's capabilities accessible via a REST API, allowing modders to live-link RTX Remix into digital content creation tools like Blender, modding tools like Hammer, and generative AI apps like ComfyUI. NVIDIA is also providing an SDK for the RTX Remix Runtime, enabling modders to deploy RTX Remix's renderer into applications and games beyond DirectX 8 and 9 Classic.

Further open-sourcing the RTX Remix platform will enable modders around the world to build even more amazing RTX remasters.

NVIDIA RTX VideoThe popular AI-powered super-resolution feature supported by Google Chrome, Microsoft Edge, and Mozilla Firefox browsers is now available as an SDK for all developers, helping natively integrate AI for upscaling, sharpening, compression artifact reduction, and high dynamic range (HDR) conversion.

Coming soon to video editing software Blackmagic Design's DaVinci Resolve and Wondershare Filmora, RTX Video will enable video editors to upscale lower quality video files to 4K resolution and convert standard dynamic range source files to HDR. Additionally, free media player VLC media will soon be adding RTX Video HDR to its existing super resolution feature.

For more information about RTX AI PCs and technology, NVIDIA at COMPUTEX.