- Luma Labs' new changes video tool Dream Machine uses AI to change video footage without re-shooting
- Characters and environments do not lose their original movements or performances
- From subtle wardrobe adjustments to complete magical scene overhauls.
Luma Labs is known for creating AI videos from scratch, but the company has the new features of the Dream Machine that allow you to completely change real video footage in subtle or explicit ways, even if it's an old home movie.
The new Modify Video Feature is for videos that you do for images like the best Photoshop tool in your video. You can change the settings, style, and even the entire character of the scene.
The company boasts that it retains everything important from the original recording, including the actor's movements, framing, timing and other important details.
The outfit you are wearing is suddenly a completely different set of clothes that you decide not to be. That blanket fort is now a ship sailing through the stormy seas, and your friend is actually a space astronaut, without using or editing the green screen.
Luma's advanced movement and combination of performance capture, AI styling, and what's called structured presets, can provide you with a full range of rethinking videos.
All you need to do is upload a video up to 10 seconds long to start. Next, select the preset from ADHER, FLEX, or REIMAGINE.
Adhesion is the most delicate option. It focuses on minimal changes, including the following garment adjustments and various textures of furniture. Flex does that, but you can also adjust the style of video, lighting, and other more obvious details. As the name suggests, Reimagine can completely remake everything about the video, take it to other worlds, remake people into cartoon animals, and send people standing on flatboards to cyberpunk hoverboard races.
Flexible AI Video
It all depends not only on the prompt, but also on the reference image and frame selection from the video if you select it. As a result, this process is much more user-friendly and flexible.
The AI video changes are not specific to Luma, but the company claims they are better than rivals like Runway and Pika due to their performance fidelity. The modified video retains the actor's body language, facial expressions and lip sync. The final result is displayed as an organic whole, not just the sewn bits.
Of course, there are limitations to the modification video tool. These are capped in 10 seconds per clip for now, making things easier to manage in terms of waiting times. However, if you want a longer film, you need to plan and solve ways to artistically incorporate different shots into one film.
Still, features like the ability to separate elements within a shot are a big deal. There may be performances that make you very happy, but in a different setting, you should be a different kind of character. Now, you can leave the performance alone and swap the garage for the ocean and the actor's feet.
Dreams for reality
It's really impressive that the AI tools can recreate a little bit of footage a little quickly and thoroughly. These tools are more than just a gimmick. AI models recognize performance and timelines in ways that feel more human than I have seen. AI models don't really understand pacing, continuity, or structure, but they are very good at mimicking these aspects.
Technical and ethical limitations prevent Luma Lab from recreating the entire film at this point, but these tools are appealing to many amateur or independent video producers. And although I don't think it's as widely used as popular photo filters, the Luma demo has some fun ideas that you'd like to try.
