Apps like Dall-E 2, Stable Diffusion, and Midjourney can transform text descriptions into detailed images. Second, ChatGPT spins a story out of prompts. Now Runway Gen-2 combines both aspects to create videos that tell stories.
But it’s a very short story, no longer than 4 seconds. And video often produces a whimsical, surrealistic effect that the prompt didn’t want. A ball of yarn that wags its tail, a cat that melts, and a horse with bicycle wheels.
Yes, we are in the early days of generative AI synthesis, but things are progressing rapidly. Runway Gen 1 was only able to apply AI effects to videos shot traditionally or generated elsewhere. From sheer magic, Gen-2 creates videos with more detail and style than competing apps like Gencraft, ModelScope Text to Video Synthesis, and Vercel.
Still, at this point, Runway is essentially a tech preview or proof of concept, far from a fully formed product capable of producing professional-level results. Also, before you start paying, you’ll be allocated credit for a 25-second video so you can try it for free. (Check the end of the article for details on the complex pricing structure.)
I used a variety of prompts to test the Gen-2 against its rivals. It started with the internet’s well-deserved cats and progressed to a more complex and detailed exposition of the subject matter, its movements, and cinematic style. Gen-2 always had the most detailed detail and generally the most stylized shots, but also some of the weirdest effects.
Runway Gen-2 test results
Instantly creating an “orange tabby cat” gives it a dreamy look with fine details in fur and whiskers, subtle and fluid movement of the cat’s face, trees swaying in the wind in the background, transitions of light and shadow. Several hi-res clips were generated with . The 16:9 widescreen size and 1080p resolution make it even more cinematic. Rival apps produce low-resolution square images with a more cartoonish or sketchy look, but Gencraft sometimes comes close with its detail and attractive composition. (Unlike other apps that create MP4 videos, Vercel generated his GIFs. However, popular video editing apps can convert GIFs to proper video files.)
But Runway Gen-2 was the one most likely to go off track. In one case, both well-shaped cats and strange tiger-colored lumps appeared. In another example, the app rendered twins Tabby. And it gets even weirder. For example, we pushed the app step by step to very complex prompts: “orange tabby cat rolling pink ball of yarn on white carpet, warm lighting, realistic.” In one attempt, part of the cat’s body disappeared. In another photo, the cat has completely disappeared, revealing a human hand from a ball of yarn. A rival app had no such hallucinations.
However, any app benefited from multiple attempts, as generative AI produced different results with each attempt. (We gave every program three shots of her in each test, rated on best attempt. We also adjusted the app to the highest quality settings for features like frame rate.)
Surrealism aside, Gen-2 has consistently impressed with its ambition. Following the cat, I tried another generated AI meme, “Astronaut on horseback.” The app created a stunning scene with a starry background, swirling dark ominous clouds, and a horse circling majestically towards the camera. The astronauts wore clean, sleek spacesuits reminiscent of SpaceX outfits, but wore traditional open-faced jockey helmets that didn’t work well in a vacuum (the other two A sealed helmet was used in one attempt). Gencraft, ModelScope videos, and Versel were much more cartoonish.
Of course, we also had to test the “Will Smith eats spaghetti” prompt that blew up the memeverse in March when the free ModelScope app program demonstrated the power of generated AI video. However, none of these apps have a sense of table manners, and depict the Fresh Prince chewing without using tools. (Cleaned up by adding “with forks” to subsequent prompts.) ModelScope created this meme, but this time it’s a lot harder. His first two attempts showed just a bowl of spaghetti, without Mr. Smith. He appeared in relative detail on his third attempt. But while Runway ML was once again far ahead in its detailed and stylized imagery, the character didn’t always exactly resemble Will Smith.
Other apps, however, have created a more animated diner, sometimes munching on what feels like his first meal in weeks, but they all show a lot of distortion on his face. I was.
Rival videos were, for the most part, far more animated than their runway counterparts, and atmospheric rather than action-packed.
In fact, rival videos were almost always much more animated than their runway counterparts, and more atmospheric than action-packed. The prompt “Elon Musk riding a horse on Mars” is extreme stillness, as what appears to be a clipping of a billionaire equestrian athlete (who bears no resemblance to Musk) slides across a barely moving background. It showed what it was like–like a high-res southern park clip. Multiple attempts were of little help, but the attempt to arbitrarily add a bicycle wheel in front of the horse was interesting.
There’s another kind of action you don’t see on the runway. it’s porn. One day, while checking through Vercel’s gallery of recent user creations, I came across a series of naked women generated by really foul-mouthed prompts (although their actions weren’t as good as the users requested). It wasn’t blatant). Her ModelScope app, which is free, also has no obvious sensors. But the runway does. I tried the Vercel gallery’s least aggressive prompt, “pretty naked girl,” on his Gen-2. We have been informed that your request appears to be in violation of our Terms of Service.
Additionally, “repeated violations may result in account suspension.” However, the app did not charge for my childish request.
AI is still inconsistent and expensive
However, it can also cost a lot of money. One obvious lesson to be learned from all these apps is that today’s AI videos are still pretty inconsistent. Some pieces work great on the first try. Others fail over and over again. Still image generators like Midjourney (and Runway’s own image generator product as well) give you low-res previews that you can choose from before paying to download. With Runway, you have to pay for the flop before you get better results.
How much does that cost? Gen-2 pricing is based on credit system, video costs 5 per second. After using your 125 free introductory credits, you need to sign up for a monthly plan starting at $15 with 625 credits that expire at the end of the month. (If you sign up for a year, the monthly fee will drop to $12 for him.)
Additional non-expiring credits can be purchased starting at $10 for $1000. None of them are very expensive if the top video speed is 4 seconds. You can make 250 for $10. However, Runway has evolved the ability to create longer movies, which can be quite expensive at current pricing.
But perhaps that day is still some time away. For now, the Runway Gen-2 only offers a fascinating glimpse of what might one day grace our TV and movie screens.
