Google unveils a shockingly good 5-second AI video generator

Google’s new AI video generator is its most advanced yet, and it could lead to more convincing deepfakes.

Google Research has announced Lumiere, an AI video generator that can create 5-second photorealistic videos from simple text prompts. What makes it so advanced is its “spatiotemporal U-Net architecture,” which “generates the entire temporal duration of a video at once through a single pass through the model,” according to the research paper.

Previous AI models created videos by generating individual images frame by frame.

In theory, Lumiere allows users to easily create and edit videos without any technical expertise. Prompts like “Panda playing ukulele at home” and “Sunset timelapse at the beach” generate detailed, photorealistic videos. You can also generate videos based on the style of a single image, such as children’s watercolor flowers.

The editing features are crazy. Lumiere can animate targeted parts of an image and use “video inpainting” to fill in blank areas in image prompts. You can also edit specific parts of your video with follow-up text prompts, such as changing a woman’s dress or adding accessories to your owl or chick video.

“Our main goal…is to enable novice users to generate visual content,” the paper concludes. “However, our technology is at risk of misuse to create false or harmful content, and we believe it is important to develop and apply tools to detect bias and malicious use cases to ensure safe and fair use.”

What the paper doesn’t mention are tools that Google is likely already developing and deploying.

At Google I/O last May, the company put safety and liability measures front and center. Google DeepMind released a beta version of an AI watermarking tool called SynthID in August, and in November YouTube (owned by Google) announced a policy requiring users to disclose whether their videos were generated by AI.

Lumiere is just research at this point, with no mention of when or how it can be used as a consumer tool. But for a company that claims “being bold with AI means being responsible from the beginning,” this is a surprising omission from the Lumiere team, assuming research is included.

Google has not yet responded to a request for comment.