Generative AI is enabling people to write extensive essays, create detailed images, and even replicate their own voices with astonishing accuracy. But when I tried out an AI-generated video service, I found that the technology is still a long way from creating compelling, cinematic videos. In fact, the whole experience was surreal.
Luma AI's dream machineis a free text video service that warns users that due to high demand, they're limited to 10 videos per day and 30 videos per month unless they pay a minimum of $29.99 per month to sign up for their starting plan. But I only had to wait a few minutes before the first prompt turned into… a very strange video.
I started with a simple request: can you generate a video of a baseball player hitting the ball out of the park?
The results were surprisingly bizarre. Instead of a smooth, realistic depiction of a home run, I saw a fever dream. The video showed an old man bending his body impossibly, preparing to throw (or catch?) a ball while simultaneously trying to swing a bat. The stadium backdrop looked reasonably accurate, but the players' movements were distorted, their uniform numbers were blurred and their faces contorted unnaturally as they moved, while the bat changed size as it was swung and the lettering on the stadium sign was incoherent.
To get more accurate results, I decided to try the prompts generated by ChatGPT. Robots can be good at talking to other robots.
The prompt described a sunny afternoon at a modern baseball park filled with cheering fans and detailed the bright team colors and white uniforms of batters with blue pinstripes. I requested a scene of a pitcher in a dark blue uniform throwing a fastball, a batter swinging horizontally, hitting a monster home run, and the crowd erupting in applause.
The results were even more perplexing. The batter appeared to be hugging himself while morphing into a strange creature. Fans were inexplicably seated close to home plate, which had transformed into an arch with a strange object resting on top of it. The batter was facing the wrong way — or was it the catcher?
Given the long-standing fears of deepfake videos and misinformation, I asked the model to show me the video. Joe Biden, Donald Trump, Pope Francisand Barack Obama The company offered to do a speech, which he declined, but agreed to do a video featuring the basketball star. Michael Jordan Give a speech in the school gym.
The video briefly shows a person who resembles Jordan before inexplicably transforming into someone completely different, while another person limps away like a zombie in ill-fitting pants. The gym setting is mostly accurate, except for the platform on which someone's leg is being chopped off, the wrong basketball symbol on the floor, and what appears to be a basketball hoop on the wall.
My Editor Matt KendrickThe former Emmy-nominated TV producer also gave it a try, first producing a thrilling historical drama set in medieval Mongolia that ended up being a somewhat disturbing reverse centaur situation.
But the software may be designed to suit a Hollywood screenplay format, such as the 2004 movie. Kal Penn/John Cho production, “Harold and Kumar Go to White Castle.” Unfortunately, when you paste in this finely crafted script, all you see is a clip of a man sitting at a desk decorated with the flag of the Belarusian pro-democracy movement and some rather phallic decorations, taking a phone call in an indecipherable language.
Text-to-video models like Luma AI and OpenAI's still-secret model Sora promise to create realistic scenes, but technical challenges seen in initial tests suggest the technology is still a long way from reality. Glitches, blurriness, and jarring inconsistencies aren't evidence of a model that's going to confuse anyone, at least not without significant improvements. So Hollywood needn't worry just yet.
The bar for success is high but not impossible. Regulators should plan ahead. If video generation technology is cheap and powerful, it could be used to trick or deceive people, or even to disrupt elections. Earlier this year, a Hong Kong bank employee was scammed out of more than $1 million. $25 million Deepfake of the company's CFO in a video call, and AI-generated recording, photograph, Avatarand Article Given the influential role AI has played in politics this year, it's only a matter of time before an AI-generated video causes a stir.
Nick ReinersThe senior geotechnical analyst at Eurasia Group said that while regulators haven't cracked down on text-to-video models, a big focus globally is on transparency, “so you know a deepfake when you see one,” a principle of the European Union's AI law, the G7's Hiroshima process and the Biden administration's executive order on AI.
Reiners believes the reluctance of major AI companies to release their models is not down to the technical maturity of their products, but rather to the negative impact they have on society. “If you look at the progress made in image generators in recent years, you can assume that we'll see similar progress in video,” he said.
In Reiners' view, the two big problems are disinformation and sexual abuse material, and he thinks the latter is likely to be addressed first. “There's a huge push from both parties to protect children.” As video models improve, deepfakes of a lewd or vulgar nature could cause an uproar before they can sway the outcome of an election.
