Startups are racing to develop the best artificial intelligence video generation tools and convince Hollywood that AI won't decimate the creative industries, but the most common early uses of AI in entertainment tend to be more mundane than de-aging Harrison Ford to simulate a young Indiana Jones.
The use cases for AI in Hollywood were drilled down in stark detail Monday during the virtual Digital Hollywood Summit, which brought together top executives at the intersection of AI and entertainment, with a focus on marketing, speeding up back-office operations and increasing revenue from existing content.
Samira Pana Bakhtiar, head of media, entertainment, games and sports at AWS, said AI presents a promising new way to make more money from many studios' largely unused back catalogs of movies and shows.
“I think the most exciting opportunity right now for our clients who have decades of archival content is the ability to search and discover that content,” Bakhtiar said.
AWS is working with other AI companies, such as Anthropic, to explore how media companies can classify archival footage by characteristics such as leading actors or sentiment analysis, helping media owners better understand what untapped value it represents to consumers.
Understanding where content can and should be distributed, creating and translating subtitles and audio descriptions, and using AI to dub audio into new languages are other applications that entertainment executives are interested in. But building the tools in-house is unthinkable for many creative content giants. That's where startups come in.
“We recognized early on, especially with this wave of generative AI, that the large foundational model technology companies were going to be incredibly important to our future, and that we weren't going to be able to invest enough in them to be able to do things alone,” said Paramount's chief technology officer, Phil Weiser.
Not only can AI help companies like Paramount to reach audiences more effectively, it can also reduce the costs of reaching audiences who might not be able to consume the media created.
For example, detailed audio description of a scene can be costly to execute well. “We hope that the use of AI will make these accessibility features more prevalent across media,” Weiser said.
Companies like Microsoft, AWS, and Nvidia are eager to incubate the next big startup, and content developers like Paramount are major customers for companies looking to revolutionize the way media is produced and distributed. These are some of the startups that AI and entertainment leaders are keeping an eye on.
Runway
Runway's co-founders are Alejandro Matamara Ortiz, Anastasis Germanidis and Cristobal Valenzuela. Runway
In its last funding round, Runway was valued at $1.5 billion, and its text-to-video conversion capabilities have led to partnerships with Paramount, AWS, Nvidia, and Google.
The focus of discussions on Monday was not on Runway's ability to produce a full movie, but rather on speeding up the process of selling a movie that has been made traditionally.
“Marketing is one area where we're seeing big adoption because you need to create derivative versions of content very quickly, and you need to do it within the workflows you're using today,” Weiser said of how Paramount is using Runway's technology.
This technology was also used in the Oscar-winning film “Everything, everywhere, all at once.”
Crikey
Krikey can also turn text into images, but it specializes in animation, for example allowing gamers to create three-dimensional avatars of themselves. This kind of technology paves the way for more prominent applications of AI in marketing and advertising.
“There's been a lot of interest in using digital human or digital character equivalents in marketing, where fans can interact with the characters,” Weiser said. Krikey is part of AWS' startup accelerator group, according to Pitchbook.
Kubrick AI
Cuebric can transform still images into multidimensional landscapes suitable for movies. The company has partnered with Nvidia and AWS for inference computing. Amazon's Bakhtar listed Cuebric alongside several other startups that use AI to create films. The company's co-founders say they want to democratize filmmaking.
Luma AI
Luma AI investor Marc Andreessen thinks AI can save comedy. Justin Sullivan/Getty Images
Luma AI, another image-generating AI startup Bakhtar mentioned, was recently backed by Andreessen Horowitz and has raised more than $68 million, according to Bain. Luma's text-to-3D image generation model, Genie, is the company's flagship product. Co-founder and CEO Amit Jain worked on Apple Vision Pro and its machine learning applications before co-founding Luma.
x Mentium
Elliot Sakhartov, a business and technology strategist for Microsoft's media and entertainment division, called xMentium a “completely unsexy” AI tool his team uses.
xMentium helps speed up the creation and referencing of legal documents and contracts. The company has raised about $7 million in funding, according to Pitchbook, and is specifically targeting the entertainment industry.
Speechmatics
Michael Kaplan, global business development head for generative AI in media and entertainment at Nvidia, mentioned Speechmatics, a UK-based startup that develops AI transcription and translation tools that can be used to create subtitles.
Speechmatics is a member of Nvidia's Inception program, a network of more than 20,000 startups.
“It really unlocks a ton of content from regions that you would never normally get,” Sakhartov said of the AI transcription and translation of subtitles.
12 Labs
Streaming services could see more ads Oleg Elkov/Shutterstock
Nvidia's Twelve Labs, which Kaplan also mentioned, uses AI to not just listen to the spoken words in your content, but to understand what's going on in the broader context. This additional context will allow it to seamlessly split content into chapters and insert ads in logical places. It will also power the ability to search videos by more ephemeral search terms like tone or action.
Sakhartov presented a hypothetical scenario in which such a tool could take a user's favorite shows and the length of their commute, then custom edit an episode to fit that time slot, including only their favorite parts.
Eleven Lab
Eleven Lab
Voice-cloning startup ElevenLabs could potentially expand distribution of movies and TV shows into new languages through AI-powered dubbing. The London- and New York-based startup became a unicorn earlier this year. Its digital replicas of voices are also used in marketing.
A digital version of Nvidia CEO Jensen Huang debuted at Computex 2024 in Taiwan in June, with the tech giant's voice generated in both English and Chinese by ElevenLabs.
1st period
The Reid Hoffman on the left was created by Hour One's AI. On the right is the real Reid Hoffman. Reid Hoffman/YouTube
Speaking of Huang, technology from Israeli startup OurOne helped animate Huang's “digital human,” renderings so lifelike that Kaplan admitted, “I had no idea. I thought it was him.”
Founded in 2019, Hour One has been exploring the possibilities of digital humans and lifelike avatars longer than most other companies. The company is also a member of Nvidia's Inception program. Among entertainment applications, the company sees opportunities to generate moving avatars of famous broadcasters for real-time news and sports updates.
