What Midjourney did with stills and ChatGPT with writing, the new app aims to do with video, spitting out animated clips based on your prompts.
Unlike full AI video editing apps like Runway Gen-1 and script-to-movie programs like InVideo, these new programs create fully composited movies instead of leveraging existing video. increase.
However, this is a very nascent field, far behind static image and text generators.
Some services limit clips to 4 seconds or less. And they often have a cartoon or hand-drawn look. Faking your friends with deepfakes on these services will soon be over. Nonetheless, these synthetic video creators represent an exciting frontier in AI creative tools that developers and artists will rapidly move forward in the months and years to come.
What are the features of the best AI video generator?
Realism and detail are key elements in synthetic video. None of these apps perfectly mimic the real world, but in some cases they come close. For example, Runway Gen-2 can also model lighting changes and fine motion. Your app should not only tackle shape, color, and motion, but also provide enough resolution to bring clarity to what you create.
Most of the time it doesn’t (resolutions range from 480p to 1080p). And the most used square format seems more suited to online postings like memes than filmmaking endeavors. Of course, at four seconds maximum, none of these clips are up to cinematic proportions, but some are even shorter, at two seconds.
However, in some cases, the synthetic video may have too many details with surreal hallucinations that are not seen in the real world or in the user-entered prompts.
I’ve seen cats whose bodies disappear and tableware that blends into people’s faces. It would be great if that was specified in the prompt, but it wasn’t. There seems to be a direct relationship between app sophistication and hallucinogenic tendencies. Programs that attempt to provide more detail may also provide surreal details.
Synthetic video has too many details that are not seen in the real world or the prompts the user inputs, with surreal hallucinations.
Whatever the quality of your app, the ability to refine your results is key. For example, how well can you make sense of complex prompts about colors, camera angles, movements, etc.? Some apps also offer filters for different styles of appearance.
But no matter how good your app is, sometimes it takes a few relationships to get things right. Generative AI produces different results each time. With this in mind, he tried each app three times for each tested prompt and chose the best one. Also, set the app to the highest quality settings (attributes such as resolution and frame rate). All of these online tools were tested with the Chrome browser.
Price matters too. One of these apps is free, while the others charge based on the length of clips created or the number of video creation prompts. Pricing can quickly become confusing, with weekly or monthly billing, annual subscription discounts, and the ability to buy more subscriptions.
Unless otherwise stated, services are priced based on the lowest possible commitment (monthly or weekly).
| product | price | Clause | format | Maximum duration (seconds) | Maximum resolution | File type | Accessible |
|---|---|---|---|---|---|---|---|
| Jencraft | 3.99/week | 25 prompts per day | square | 2 | 720p | MP4 | web, mobile |
| model scope | free | N/A | square | Four | 720p | MP4 | web |
| runway ML | $15/month (625 credits) | 5 credits per second | 16:9 | Four | 720p | MP4 | web |
Runway Gen-2 is the company’s first product to create fully synthetic AI videos. (Gen-1 uses AI to modify existing videos.) So while it shows a lot of promise, it also has a lot of quirks. The app produces detailed and great-looking clips in 16:9 widescreen format at 1080p resolution with sophisticated effects. For example, in our cat movie, you can see the cat’s whiskers flickering, trees swaying in the wind, and lights and shadows dancing on his face. The images aren’t very realistic, but rather sophisticated renderings, sort of somewhere between live footage and cutscenes from the best video games.
The app understands not only ‘orange tabby cat’, but also complex prompts such as ‘orange tabby cat rolling pink ball of yarn on white carpet, warm lighting, realistic’. . Other rival apps, such as Gencraft, also understand these specs, but render less detail and sophistication.
Perhaps a side effect of how ambitious Runway Gen-2 is, it’s also possible that its creativity has gone too far. The app was prone to intense hallucinations that rivals didn’t. For example, in one attempt to render a tabby cat, the cat did not exist, but a ball of yarn sprouted in a human hand. In another experiment, horses won bicycle wheels. However, generative AI is a game of trial and error, and we typically got pretty good results within 3 trials. Even with our best attempts, however, we sometimes experienced mild hallucinations, like an otherworldly glowing white sphere floating between the cat and the ball of yarn.
Each of these silly deeds has a cost, but not too much. Gen-2’s pricing is based on credits, of which he buys 5 of her a second’s worth of video. A new user gets 125 free credits before he upgrades to a monthly plan. Prices start at $15 for 625 credits and expire at the end of the month. Additional permanent credits he can purchase starting at $10 per 1000.
Gencraft is one of the pioneers in the emerging field of pure text-to-video generators. But that in itself has its limits. For example, the app was adept at rendering an orange tabby cat, and was able to figure out advanced modifiers within its prompts, like adding a white carpet or a pink ball of yarn (and a relatively fast can render a 2 second video in less than 1 second) 30 seconds. ) But getting cats right is almost a prerequisite for any online creation tool. Gencraft struggled with new Internet memes like “Astronaut on horseback” and “Will Smith eating spaghetti.” These images were blocky or distorted. For example, Smith’s head appeared to be reshaped and his eyes were bulging.
Gencraft offers several tools for fine-tuning the results. Choose from 31 style collections. Our tabby looked radically different in cartoon, oil painting, abstract and photorealistic options. (If you choose not to select any style at all, “Real” is a bit redundant, as images tend to be that way.) You can also specify keywords for topics or attributes you want to exclude from your video, such as “cross.” can. – eyes” or “blurred background”.
Gencraft has some additional limitations. Limited to 2 second videos displayed in square format. Rival Runway Gen-2 and free ModelScope support up to 4 seconds. Runway also offers 16:9 wide format and 1080p resolution (720p on Gencraft).
Gencraft offers a free tier that allows 10 video creation prompts per day, but the clip contains a watermark. You can upgrade to 25 unwatermarked prompts per day for $3.99 per week, or unlimited prompts for $9.99 per week.
ModelScope Text to Video Synthesis became an internet sensation when it was used to generate the “Will Smith eating spaghetti” video meme. And there are plenty of opportunities to try out your own crazy ideas. Hosted on the AI developer hub Hugging Face, ModelScope is free and allows you to create videos up to 4 seconds long for this emerging technology.
Of course, “free” has its drawbacks. Subject to the whims of the site’s servers, they are often overwhelmed and fail to complete the job on the first (or second, or third) attempt. Turning the tool to the highest quality setting (like frame rate) makes it even less likely. Sometimes you have to wait until quieter times to get things done. Also, Hugging Face does not provide storage. You should download the video as soon as you create it. (They may disappear due to server malfunction, so please wait.)
ModelScope’s clip doesn’t have the high-end detail of Runway Gen-2’s work (although it also lacks Runway’s hallucinatory imagery). But it does have a high-quality cartoon-like quality that rivals his Gencraft and Vercel from paid providers. The depiction of the orange tabby cat was attractive. Like all his ModelScope subjects, the cat was very fidgety and jumping all over the place. Will Smith was also quite enthusiastic, stuffing pasta into his mouth with his bare hands. However, although cartoonish, the subject was very recognizable as the famous actor. Gencraft’s equally fidgety version was more skewed. Runway Gen-2 created a dreamy, surreal version that looks cool but doesn’t look much like Smith.
The support from the developer community and the zero price of ModelScope make it worth keeping up with the rapid progress of this technology.
Want to learn more about using AI in your creative work? Here are the 5 best AI image generators for text-to-video conversion.
