AI startup Runway has released its first mobile app on iOS. This allows people to use Gen-1, the company’s video-to-video generative AI model, directly from their mobile phones. You can download the app here. Free users are offered a limited number of credits.
Gen-1 allows you to convert existing videos based on text, image or video input. Functionally, it works like the Style Transfer tool (but unlike Style Transfer, it doesn’t apply a filter, it produces an entirely new video as output). For example, you can upload a video of someone cycling in a park and apply an aesthetic or theme to it. You can give your video the look of a watercolor painting, a charcoal sketch, and more.
Of course, this is generative AI, so the output is often… weird. For example, if you add a claymation effect, the resulting model will not work like this: real claymation. The model warps between each frame. The limbs stretch and contract. Features will melt and become dirty. However, it’s to be expected and doesn’t take away from the fun.
Here, for example, three different renderings of Al Pacino’s iconic clip. heat (1995). Most notable to me is the bottom right clip. This uses a photo of the cat I took as an intermediary. , gave the hands a little fur. The other two clips in the top row are preset filters.
Here’s another example: A video of St. Paul’s Cathedral in London with the Paper and Ink filter applied. Not an amazing effect, but very easy to make. And I’m sure it would be great in the hands of more experienced and creative individuals.
I’ve been testing Runway’s app for a few days now and it certainly made the whole process of creating this kind of video smoother. (Runway’s primary software suite is available on the web, increasing the distance between capturing and producing footage.) Of course, this isn’t a seamless experience. Expect the usual inefficiencies and unexpected errors in your app’s first release. But as his CEO of the runway, Cristobal Valenzuela, said, The Bargeit is important to make these tools mobile.
“That’s why a phone makes a lot of sense, because you record directly from the device and tell the Gen-1 how to convert that video,” says Valenzuela.
There are other limitations worth mentioning. Footage longer than 5 seconds cannot be manipulated and there are certain prohibition prompts. For example, you cannot generate nudes. Also, it seems that copyrighted works are off limits. My prompt to make a video “in the style of a Studio Ghibli movie” was rejected. This doesn’t sound like much, but it feels like an instant era of his mobile editing era. The processing happens in the cloud and may get faster over time. The app currently only supports Runway’s Gen-1 model, but Valenzuela says his Gen-2, which is purely generative, will be added soon.
However, what these notes do not fully capture is the possibility of such tools. The AI text-to-image model output was also blurry and unrealistic at first. Now they are used to fool the public with flashy pictures of the Pope.
Valenzuela is compared From the current era of generative AI to the 19th century “optical toy” stage, scientists and inventors have created all sorts of devices whose functions were trivial, but modern It was also the ancestor of the camera of Runway’s mobile app feels his one of those toys. I doubt it will ever be used in professional production, but I can’t imagine how great a tool like this could be in the future.
