A group of Adobe researchers recently published a paper on a new generative AI model called VideoGigaGAN, which they believe could be featured in future products. Upscale low-quality videos up to 8x their original resolution without sacrificing stability or important aspects of your source material. His website for the project has some demo clips showing its features. You can convert a blurry waterfall video with a resolution of 128×128 pixels to footage with a resolution of 1,024×1,024 pixels.
Posted by @luokai
View in thread
The remarkable thing about AI is that it doesn't skimp on the details. The human subject's face shows skin texture, wrinkles, hair strands, etc. Other demos have a similar level of quality. Thanks to this technology, we will be able to better identify swans swimming in ponds and flowers on trees. It may seem strange to focus so much on skin wrinkles and feathers. However, this level of detail needs to be established if companies like Adobe aim to implement image enhancement AI at scale.
AI improvements
You probably have some questions about the latest project on the platform, including how it works. Hmm, that's complicated.
“GAN” in VideoGigaGAN is an abbreviation for Generative Adversarial Network, which is a type of AI that can create realistic images. Adobe's version is based on his GigaGAN, which specializes in upscaling not only actual photos but also generated content. As TheVerge points out, the problem with this technology is that it doesn't improve video quality without introducing multiple issues like strange artifacts. To solve this problem, Adobe researchers used a variety of techniques.
The research paper explains the entire process. Although the content is thick, you can read it on your own to understand the whole picture. Essentially, it introduces a “flow-guided propagation module” to ensure frame-to-frame consistency in the video, anti-aliasing to reduce artifacts, and a “high-frequency feature shuttle” to compensate for sudden drops in detail. did. VideoGigaGAN has a lot more features than what I just mentioned, but that's the gist of it.
potential inclusion
Will this be included in future Adobe products or rolled out as a standalone app? Probably, at least that's what we think.
Over the past year, the company has focused on implementing artificial intelligence into its software, from the launch of Firefly to the new assistant in Acrobat. A few months ago during Adobe MAX 2023, a video upscaler called Project Res Up was previewed at the event, and its performance was similar to what you see in the VideoGigaGAN demo. Old movies from the 1940s ran at a resolution of 480 x 360, now it's a crisp 1,280 x 960. The blurry image of the elephant in the river became very clear. The presenter also mentions how the software can upscale clips to his four times their original quality.
Admittedly, this is speculation, but it's entirely possible that VideoGigaGAN is the engine behind Res-Up. Adobe's future products could give people a way to upscale old family videos and low-quality footage into the movies we envision in our heads. Perhaps the recent preview is a hint of an impending release.
VideoGigaGAN is still in development, so it is unclear when it will be released. There are several obstacles in your way. The AI can't properly handle videos longer than 200 frames or render small objects, but we'll keep an eye on it.
In the meantime, check out TechRadar's list of the best AI image upscalers of 2024.
