× close
We present VideoGigaGAN, a generative video super-resolution model that can upsample videos containing high-frequency details while maintaining temporal consistency. Above: Comparison of TTVSR consistency and the approach using BasicVSR++. Our method produces more fine-grained and temporally consistent videos than previous methods. Bottom: Our model can generate 8x super-resolution high-quality videos. credit: arXiv (2024). DOI: 10.48550/arxiv.2404.12388
A team of video and AI engineers at Adobe Research has developed an AI application called VideoGigaGAN. This can take a blurry video and enhance it into a more well-shaped product. The team will discuss their work and results. arXiv Preprint server. Also, his website page for the project features some examples of enhanced videos.
AI applications have been in the news a lot lately, mainly due to the release of LLMs such as ChatGPT that consumers can use to generate a variety of outputs. But AI research is also underway in other areas, such as creating artificial images and videos.
In this new effort, a team at Adobe has created an application (also known as upscaling) that accepts blurry video samples and returns the same samples with significantly improved sharpness and clarity after processing.
This is called VideoGigaGAN. The name comes from the previously demonstrated app GigaGAN, which generated new photos and improved old ones. GAN stands for Generative Adversarial Network.
As the name suggests, the team uses a generative adversarial network to teach the system what crisp, clear video looks like (like individual eyebrows, not blurry blobs), and then , adding a “flow-guided propagation module” to make it consistent between video frames.
They also used anti-aliasing techniques to prevent what they describe as “AI weirdness” and high-frequency feature shuttles to address unexpected video quality degradation.
The researchers say the result is a system that can improve video quality by up to eight times, but all of this leaves them with strange colors and unevenness in AI-generated images and videos. lines and other well-known issues.
They admit that some of the output is completely artificially generated based on estimates made by the system to fill in missing images. For example, skin pores, lines around the eyes, and even eyelashes are added, giving the resulting video a crisp and crisp quality.
The team notes that at this time, the announcement of this system is a demonstration and not a pending release. Therefore, it is unclear whether Adobe will release this to the general public.
For more information:
Yiran Xu et al., VideoGigaGAN: Towards detailed video super-resolution, arXiv (2024). DOI: 10.48550/arxiv.2404.12388
Project website page: videogigagan.github.io/
Magazine information:
arXiv
© 2024 Science X Network
