Video super-resolution (VSR) is a classic but difficult task in computer vision and graphics that attempts to recover high-resolution video from low-resolution video. VSR faces two significant problems. The first challenge is to ensure temporal consistency across output frames. The second challenge is to create high frequency information in the upsampled frame. Previous techniques have addressed the first hurdle and demonstrated good temporal consistency in upsampled videos.
However, these methods often result in blurry images and cannot provide high-frequency appearance details or realistic textures. An effective VSR model must generate reliable new content that is not present in the low-resolution input video. However, his current VSR model has limited generation capabilities and cannot create detailed looks.
Generative Adversarial Networks (GANs) have demonstrated superior generative capabilities in image super-resolution. These approaches can accurately simulate the distribution of high-resolution images and produce fine-grained details in upsampled images. GigaGAN enhances image super-resolution model generation capabilities by training large-scale GAN models on billions of images.
The dawn of VideoGigaGAN
Adobe researchers have created VideoGigaGAN, a new generative VSR model that can generate videos with high-frequency features and temporal consistency. VideoGigaGAN is built on his GigaGAN, a large-scale image upsampler.
Adding a time module to the GigaGAN video model results in severe time flickering. Adobe uncovers a number of critical issues and provides approaches to improve the temporal consistency of upsampled videos.
Learn more at VideoGigaGAN
According to the researchers' findings, VideoGigaGAN produces temporally consistent videos with finer-grained appearance features than previous VSR techniques. The authors compared VideoGigaGAN to his state-of-the-art VSR model on a public dataset and verified its effectiveness by displaying video results at 8x super-resolution.
in paper Published on April 18, Adobe said VideoGigaGAN is superior to previous video super resolution (VSR) approaches because it can provide more granular features without introducing “AI weirdness” to the footage. claimed to be.
Top-notch image or video quality
In the simplest terms, generative adversarial networks (GANs) have been successful in upscaling still images to high resolutions, but are unable to do so with videos without introducing flickering or other undesirable artifacts. is difficult. Other upscaling methods can prevent this, but the results will be less sharp and detailed. VideoGigaGAN promises to offer the best of both worlds: less flicker and distortion across the output frame, and better image/video quality of GAN models. The company provides a variety of examples demonstrating its full-resolution efforts.
Since this is just a research preview, we don't know if Adobe will make VideoGigaGAN available to consumers through Creative Cloud software such as Premiere Pro. During his MAX event in October 2023, the company revealed a diffusion-based upsampling experiment called Project Res-Up that improves the quality of his low-resolution GIFs and video recordings.
