OpenAI's Sora may be facing its biggest rival yet: Luma's artificial intelligence video generator Dream Machine, which has caused wait times of several hours due to high demand for the debut of a free public beta version of its software.
The startup, which is backed by Silicon Valley venture capital firm Andreessen Horowitz, said the sheer volume of traffic is increasing video processing times but that it will continue to increase the capacity of its software, the company confirmed.
(Photo by John Schnobrich on Unsplash)
As of January 2024, this little-known tech startup has reportedly raised over $70 million, notably $43 million in Series B. The startup launched Genie 1.0 in November 2023, a model for generating 3D assets from text, which is an addition to the startup's impressive AI video generator.
Well-known AI video creators and filmmakers have been able to test Dream Machine's ability to create videos from text prompts and still images before the public beta is available. Early responses have been positive, and those who are just getting started seem very impressed. It's been compared to OpenAI's Sora, but some say it's already superior.
Tests from other sources have shown that the text-to-video feature only occasionally depicts what is asked in the prompt. The film was produced in just a few minutes and features high-definition, highly detailed material and incredibly smooth, non-jerky action.
Read also: MIT unveils new algorithm that can learn languages just by watching videos
Chinese short-video apps join AI video generation race
China's second-largest short-video app, Kuaishou, has recently entered the race to develop the best AI video generator with its app, Kling, which is said to use text clues to create high-quality videos.
Kling is currently in an experimental stage. It can convert text input into 1080p video clips up to 2 minutes long. According to the makers, Kling can generate realistic and fantastical scenes and supports a variety of aspect ratios. The demo movie showed different scenarios, such as a boy eating a hamburger and a white cat running through the city.
Google's Lumiere
Earlier this year, Lumiere was one of the earliest AI video generation programs released by Google. This Google Research project introduced a new Space-Time U-Net architecture that can generate the duration of an entire video in a single model pass.
Unlike other video models that generate remote keyframes and then apply temporal super-resolution, Lumiere uses a unique technique that makes global temporal consistency more achievable.
The architecture combines temporal and spatial downsampling and upsampling with a pre-trained text-to-image diffusion model, enabling Lumiere to process data at a range of spatial and temporal scales and natively output low-resolution video at full frame rate.
The groundbreaking spatio-temporal U-Net architecture is capable of generating full frame rate video clips for a variety of applications, from image-to-video conversion, video inpainting to styled content creation.
Related article: F1 unveils AI-generated Montreal Grand Prix winner's trophy, created by Amazon
ⓒ 2024 TECHTIMES.com All rights reserved. Please do not reproduce without permission.
