It feels almost archaic, remembering the days when AI could only generate images from text prompts. Generative AI has gotten more and more powerful over the past few years, making the leap from photography to video with the advent of tools like Sora. And now Microsoft has introduced the most impressive (and terrifying) powerful tool we've ever seen.
VASA-1 is an AI image-to-video model that can generate videos from a single photo and audio audio clip. The video features synchronized facial and lip movements, and features “a wide range of facial nuances and natural head movements that contribute to the perception of authenticity and vibrancy.”
Microsoft explains how the technology works on its research website. “The core innovations include generative models of global facial dynamics and head movements that work in the facial latent space, and the use of video to create such an expressive and disentangled facial latent space. Through extensive experiments, including evaluation on a set of new metrics, we demonstrate that our method comprehensively outperforms previous methods in various aspects. Our method not only provides high-quality videos with realistic face and head dynamics, but also supports online generation of 512×512 videos at up to 40 FPS with negligible start-up delay. It enables real-time interaction with lifelike avatars that emulate human conversational behavior.”
In other words, deepfake videos can be created based on a single image. It's worth noting that Microsoft claims the tool is a “research demonstration and there are no plans for a product or API release.” Seemingly trying to allay fears, the company has indicated that the VASA-1 won't be in the hands of users anytime soon.
From Sora AI to Will Smith eating spaghetti, we've seen all kinds of weird and wonderful (but mostly weird) AI-generated video content, and it's getting more and more real. Look at how much the generation AI has improved in his one year.
