Software giant Adobe has introduced a new AI feature known as VideoGigaGAN. It promises to upscale videos to 8x their original resolution while minimizing common visual artifacts. This means you can convert 720p HD resolution videos to 4K with little loss in quality.
The rise of AI in video editing
Adobe has been integrating artificial intelligence into its products at an impressive pace. Some particularly impressive features include AI-driven object addition and removal in Premiere Pro and text-based image generation in Photoshop.
VideoGigaGAN is the latest in Adobe's suite of AI tools. This new tool improves the clarity and detail of your videos without introducing common drawbacks like flickering and distortion.
This technology leverages advances in General Adversarial Networks (GAN), a deep learning architecture that consists of two distinct neural networks: a generator and a discriminator. The generator's role is to create data so convincing that it can be mistaken for real data from the target distribution, while the discriminator's job is to distinguish between the generator's fake data and the actual real data. (or training data). The two neural networks are in conflict with each other. Through competition, the generator learns to produce more realistic data in order to outperform the discriminator, and the discriminator becomes better able to detect the subtleties that distinguish the real from the fake.
GANs have pushed the boundaries of image, video, and audio AI generation. VideoGigaGAN is the next version of this exciting field, using advanced deep learning models to “fill in the blanks” by adding many new pixels to high-end videos. It excels at both adding sharpness and preserving detail, something that previous models couldn't achieve at the same time.
Understanding Video GigaGAN
VideoGigaGAN's secret sauce is a combination of “temporal attention,” a system that reduces artifacts that accumulate over time, and neural networks, including feature propagation, which is responsible for adding details that weren't previously present. . Sprinkled on top of that is anti-aliasing and a feature called “HF Shuttle” that enhances high-frequency capabilities.
The results were impressive. Low-resolution videos look sharper when processed by VideoGigaGAN. Even things that are considered difficult, such as hair and skin texture, can be successfully applied.
This is pretty cool, but at the end of the day, it's still just generative AI. What this means is that the added details do not exist in reality. These are newly added pixels predicted with high confidence by AI. Maybe the AI will color and add small pimples on your face that don't exist. That's certainly a possibility. With this in mind, this type of technology would never stand up in court as forensic evidence. Just in case you're feeling a CSI vibe (I certainly was).
Despite its limitations, this technology represents an exciting step forward. Video producers may soon be able to transform old, low-quality footage into high-resolution productions with enhanced detail. Adobe isn't alone. There is also competition among software developers like GAN, with NVIDIA and Microsoft hard at work on their own AI upscaling technology.
One thing is clear. Thanks to AI, the future of video editing looks brighter than ever.
VideoGigaGAN was described in a recent study published on a preprint server. arXiv.
Thank you for your feedback!
