Don't miss leaders from OpenAI, Chevron, Nvidia, Kaiser Permanente, and Capital One, only at VentureBeat Transform 2024. Gain key insights about GenAI and grow your network at this exclusive three-day event. Learn more
New York City-based Runway ML (also known as Runway) is one of the earliest startups focused on realistic, high-quality generative AI video creation models.
But since the debut of its Gen-1 model in February 2023 and its Gen-2 model in June 2023, the company has been usurped by other highly realistic AI video generators, including OpenAI's unreleased Sora model and Luma AI's Dream Machine model, which was released last week.
That's changing today, as Runway launches a major counterattack in the generative AI video wars. Today, the company announced Gen-3 Alpha, which, according to a blog post, is “the first in a series of upcoming models trained by Runway on its new infrastructure built for large-scale multi-modal training” and a “step toward building general-world models” — AI models “capable of representing and simulating a wide range of situations and interactions that you might encounter in the real world.” Check out the example videos Runway created with Gen-3 Alpha at the bottom of this post.
Gen-3 Alpha allows users to generate high-quality, detailed and highly realistic 10-second video clips with a wide range of emotional expressions and camera movements with high accuracy.
Countdown to VB Transform 2024
Join enterprise leaders at our flagship AI event in San Francisco July 9-11. Network with your peers, explore the opportunities and challenges of generative AI, and learn how to integrate AI applications in your industry. Register now
According to an email from a Runway spokesperson to VentureBeat, “This initial rollout will support 5- and 10-second generation and will significantly reduce generation times; 5-second clips will take 45 seconds to generate, and 10-second clips will take 90 seconds to generate.”
An exact release date for the model has yet to be announced, with Runway only releasing a demo video on the X's website and social accounts, and it's unclear whether the model will be available on Runway's free plan or if a paid subscription (starting at $15 per month or $144 per year) will be required to access it.
After this article was published, VentureBeat spoke with Runway co-founder and CTO Anastasis Germanidis, who confirmed that the new Gen-3 Alpha model will be available to Runway’s paid subscribers first in the coming “days,” but that the free tier is on track to get the model at a future date to be announced.
A Runway spokesperson echoed the announcement, telling VentureBeat in an email: “Gen-3 Alpha will be available soon and will be available to Runway paid subscribers, Creative Partners program, and Enterprise users.”
On LinkedIn, Runway user Gabe Michael said he expects it to be available later this week.
Regarding the X, Germanidis wrote that Gen-3 Alpha “will soon be available on Runway products and will enhance all the existing modes you're used to using (text-to-video, image-to-video, video-to-video) as well as several new modes previously only available on the higher-performance base model.”
Germanidis also wrote that since releasing Gen-2 in 2023, Runway has learned that “video diffusion models are nowhere near saturating the performance gains that come with scaling, and these models build very powerful representations of the visual world by learning the task of predicting videos.”
Diffusion is the process of training an AI model to learn concepts from annotated image/video and text pairs and reconstruct visuals (still or video) of the concept from pixelated “noise.”
Runway said in a blog post that Gen 3-Alpha was “jointly trained on video and imagery” and is “a collaborative effort from an interdisciplinary team of researchers, engineers, and artists,” but the specific dataset has not yet been made public. This follows a trend among most other major AI media generators of not disclosing exactly what data their models were trained on, let alone whether the data was sourced through paid licensing agreements or simply scraped from the web.
Critics argue that AI modelers should pay the original creators of the training data through licensing agreements, and have even filed copyright infringement lawsuits on the matter, but AI model companies generally take the position that they are legally allowed to train on any publicly available data.
When asked what training data was used for Gen-3 Alpha, a Runway spokesperson emailed VentureBeat saying, “We have an in-house research team that oversees all training, and we train our models using curated in-house datasets.”
Interestingly, Runway also says that it is already “working with major entertainment and media organizations to create custom versions of Gen-3,” which “allows for more stylistically controlled and consistent characters, targeting specific artistic and narrative requirements, and more.”
Although no specific organizations were mentioned, so far, Everything, everywhere, all at once and The People's Joker He revealed that Runway was used to create effects for some parts of the film.
Runway posted a form in its Gen-3 Alpha announcement inviting other organizations interested in getting a custom version of the new model to apply here. The cost of training a custom model was not disclosed.
Meanwhile, it’s clear that Runway isn’t giving up in the fight to be a leading player or leader in the rapidly evolving field of AI-generation video production.
Source link
