Just days after Kuaishou Technology, a Chinese company known for its short video platform, released its text-to-video model Kling, a new competitor has emerged. Luma AI, a US-based startup focused on visual AI, has announced a new video generator similar to OpenAI's Sora. Luma AI has announced a new tool called Dream Machine.
Unlike Sora and Kling, Dream Machine is available to the public. According to the company, Dream Machine is a next-generation video model that uses AI to create high-quality, realistic shots from natural language prompts. “Dream Machine is an AI model that quickly creates high-quality, realistic videos from text and images,” reads the introduction on the official website.
According to Luma AI, Dream Machine is a highly scalable and efficient Transformer model trained directly on video, capable of generating physically accurate shots. The company claims that Dream Machine is the first step towards building a universal imagination engine. The tool is now available to anyone.
The company claims that Dream Machine can generate 120 frames in 120 seconds, allowing for faster iteration. The tool is reported to generate smooth motion, cinematography, and dramatic five-second shots. Aside from the tools, it can also understand how humans, animals, and objects interact with the physical world, allowing you to create videos with character consistency and accurate physics. The tool also helps you experiment with an endless array of fluid, cinematic, and natural camera motions.
Dream Machine is not without limitations: currently, the official website lists morphing, movement, text, and Janus as limitations.
Sora uses natural language prompts to generate high-resolution videos up to one minute long, while Kling can create videos up to two minutes long. Dream Machine is free to use, so I signed up and tried to create a short video. The prompt used was “Peter Pan flying on an intergalactic carpet.” It took about an hour to create the video. The output was strange, with Peter Pan floating in a dress with his fingers bent and no carpet.
© IE Online Media Services, Inc.
First uploaded: 13 Jun 2024 16:22 IST
