Adobe plans to bring the Firefly generated AI (genAI) video model to its video editing app Premiere Pro later this year, and will also provide access to third-party AI video creation tools such as OpenAI's Sora.
Adobe on Monday gave a glimpse of its Firefly video AI model in action, along with new features coming to Premiere Pro.
There is an “Add/Remove Object” that allows the user to replace or remove items from the scene. Here, the AI ”Smart Him Masking” tool allows users to select objects to change the color of the actor's tie, remove the microphone from his boom or his mug of branded coffee. etc., or remove them completely from the shot. Medieval fantasy show.
According to Adobe, the Text-to-Video tool lets you create new content in Premiere Pro to help create supplemental B-roll footage.
Finally, generation extensions add new frames to clips, allowing editors to keep shots longer. “Additional media can be used to fine-tune edits, hold additional beats, cover transitions, and more,” an Adobe spokesperson said in a briefing ahead of the announcement. .
Adobe says Premiere Pro's genAI feature will be available in beta by the end of this year when the Firefly AI video model launches.
“Generative AI models are becoming increasingly multimodal, incorporating modalities such as images, audio, and video for both inputs and outputs,” said Arun Chandrasekaran, Special Vice President Analyst at Gartner. states. Existing text-to-video capabilities are “in their infancy today,” but the potential is “immense.”
Other AI video creation models are also gaining traction, most notably Sora, but Adobe seems intent on building its own capabilities. According to a recent Bloomberg report, Adobe pays photographers and artists about $3 per minute for video content that can be used to train AI models.
Forrester senior Nikhil Lai said Adobe will face competition from the likes of OpenAI, but will also benefit from its position as an incumbent in the market.
“Many editors have fond memories of the look and feel of Adobe tools, so we expect to see demand from editors for access to Adobe's AI capabilities,” Lai said. .
Adobe also revealed plans to partner with other vendors and include third-party models in Premiere Pro. This includes Sora as well as models from Pika Labs and Runway. These are just the beginning, an Adobe spokesperson said on a briefing call.
“Customers told us they wanted choice, and we see a future with thousands of different customized models, each powerful in its own niche,” he says. I did.
Adobe has announced that it will apply its existing “Content Credentials” functionality to assets generated by third-party models. This provides information about how the video was modified using AI and which model was used.
However, using multiple third-party tools can increase the risk of producing trained assets based on copyrighted material. To help Premiere Pro users avoid adding copyright-infringing content to their videos, the menu of available third-party modes now includes a green check mark to indicate modes that are considered commercially safe. Masu. But customers still need to do their own due diligence if they want to access other vendors' AI models, an Adobe spokesperson said.
Adobe said the work to integrate third-party tools is “exploratory” at this time, with no start date set yet.
