Luma AI Launches Dream Machine, an AI-Powered Text-to-Video Generative Platform

Luma AI on Wednesday globally released its artificial intelligence (AI) text-to-video generation model, Dream Machine. The platform can generate videos of up to five seconds in length from simple text or descriptive text prompts. The AI ​​model can generate videos in a variety of styles, including cinematic, animated, and realistic. The AI ​​company says Dream Machine is fully trained on video and can generate “physically accurate, consistent, and event-rich shots.” The platform is currently free to access and use, but there appears to be a daily generation limit.

Luma AI Launches Dream Machine

According to its website, Dream Machine's AI model is built on a Transformer model and was trained directly on videos. Typically, large language models (LLMs) are trained on text and images, and then moved to videos, as they require deeper spatial and motion understanding. The company added that “Dream Machine is the first step towards building a universal imagination engine.”

Dream Machine joins video generation platforms such as Runway AI and Pika 1.0, which are also available in the public domain and offer 3-5 second video generation. Gadgets 360 trialed the platform and found that the platform's prompt compliance is substandard; it struggles with multiple characters or when the prompt is too complex. However, compared to the other two, it is able to generate higher quality cinematic videos.

The AI ​​platform takes 120 seconds to generate a video with 120 distinct frames, according to the company. Dream Machine also understands how humans, animals, and objects interact with the physical world, said to create videos with accurate physics and character consistency.

But Luma AI also highlighted some limitations in the current mode, including movement, text, morphing, and the famous Janus problem, where instead of showing a consistent 3D output, the AI ​​model shows multiple standard views of an object from different directions.

Luma AI's Dream Machine creates a video about Stormtroopers
Photo credit: Luma AI/Dream Machine

Luma AI has not released any technical details about its AI model, so we do not have any information about parameter sizes, benchmarks, architecture, training methods, nor details about sourcing the training data. Of note, Gadgets 360 was able to generate multiple videos containing copyrighted characters.

To try out this platform, visit the website and click on the “Try Now” button. Users need to sign up to the platform before they can generate videos.