Editor's Note: This post was AI Decode Serieswill demystify AI by making the technology more accessible, and showcase new hardware, software, tools and acceleration for RTX PC and workstation users.
Video is everywhere. Today, approximately 80% of Internet bandwidth is used for streaming video from content providers and social networks. Although screens are getting bigger and supporting higher resolutions, almost all video quality is 1080p or below.
The upscaler helps to sharpen streamed video, leveraging the AI of the NVIDIA RTX platform to significantly improve image quality and detail.
What is an upscaler?
The large file size of video makes it harder to compress and transmit than images or text, and platforms like Netflix, Vimeo, and YouTube get around this limitation by encoding video, the process of compressing the raw source of the video into a smaller container format.
The encoder first analyzes the video and determines what information it can remove to meet the target resolution and frame rate. If the target bitrate is insufficient, the video quality will be degraded, losing detail and clarity and introducing encoding artifacts. Smaller files are easier to share on the Internet, but they will look worse.
Typically, software on the viewer's device will upscale the video file to match the native resolution of the display. However, these upscalers are very simple and only multiply pixels to meet the required resolution. Although they can help sharpen the edges of objects and scenes, the final video usually contains encoding artifacts and can look unnatural and oversharpened.
AI knows a better way
The NVIDIA RTX platform uses AI to easily remove video artifacts and upscale.
The process of AI upscaling analyzes images and motion vectors to generate new details not present in the original video. Rather than simply adding more pixels, it recognizes and enhances patterns in the image to provide more detail and video quality.
Before processing begins, we first need to remove image artifacts. Artifacts – unwanted distortions and anomalies that appear in video and image files – are caused by overcompression and data loss during transmission and storage.
The NVIDIA AI network helps remove image artifacts and eliminate blocky areas sometimes seen in streaming video. Without this first step, the AI upscaler may end up enhancing the artifacted image itself instead of the desired content.
Extra Large Video
Just as putting on prescription glasses makes the world instantly clearer, RTX Video Super Resolution, one of NVIDIA's latest innovations in AI-enhanced video technology, helps you see the world in streaming videos more clearly.
Available on GeForce RTX 40 and 30 Series GPUs and RTX Professional GPUs, it uses AI running on dedicated Tensor Cores to remove block compression artifacts and upscale lower-resolution content up to 4K to match the user's native display resolution.
RTX Video Super Resolution enhances every video you watch in your browser. Artifact removal combined with AI upscaling technology makes even low-bitrate Twitch streams look amazingly clear. RTX Video Super Resolution is also supported in popular video apps like VLC, allowing users to apply the same upscaling process to offline videos.
Creators will soon be able to use RTX Video Super Resolution in editing apps like Black Magic's Davinci Resolve, making it easier than ever to upscale lower-quality video files to 4K resolution or convert standard dynamic range source files to high dynamic range (HDR).
Welcome to High Dynamic Range
RTX Video now also supports AI HDR. HDR video supports a wider range of colors and adds greater detail, especially in the dark and light areas of an image. The problem is, there isn't a lot of HDR content online yet.
Enter RTX Video HDR. Simply turn it on and our AI network will convert your standard or low dynamic range content into HDR, performing correct tone mapping so images look natural and preserve pristine colors.
Introducing AI to every field
RTX Video is the latest implementation of AI upscaling, powered by NVIDIA RTX.
Members of the GeForce NOW cloud streaming service can play their favorite PC games on nearly any device: GeForce RTX servers around the world first render and encode the game's video content, then stream it to the player's local device, just like streaming video from any other content provider.
Members with older NVIDIA GPU-powered devices can also use AI-enhanced upscaling to improve the quality of their gameplay. This means you can enjoy the benefits of both server-rendered gameplay with RTX 4080 class GPUs in the cloud and AI-enhanced streaming quality. Learn more about how to enable AI-enhanced upscaling on GeForce NOW.
NVIDIA SHIELD TV takes this one step further, using AI neural networks processing directly on the NVIDIA Tegra system-on-chip to upscale 1080p quality or lower content from nearly all streaming platforms to the display's native resolution. This means that with the push of a button on your remote, users can enjoy improved video quality for content streamed from Netflix, Prime Video, Max, Disney+ and more.
SHIELD TV is available now for discounts of up to $30 in North America and £30 or €35 in Europe as part of Amazon's Prime Day event running from July 16-17. European Prime members with an eligible SHIELD TV purchase will receive a free month of GeForce NOW Ultimate membership, allowing you to stream GeForce RTX 4080 class PC gameplay directly to your living room.
AI has improved video quality like never before, setting a new standard for the streaming experience.
Generative AI is transforming games, video conferencing, and interactive experiences of all kinds. To learn more about the latest developments and where they're headed, AI Decode Newsletter.
