Image credit: Pawel Czerwinski
A new tool from Google is challenging the world's film and TV composers by automatically generating the perfect music for footage.
Google's DeepMind lab has unveiled another revolutionary AI tool. The lab unveiled V2A (Video to Audio), a groundbreaking technology that automatically syncs audio to video.
V2A can analyze video content and infer its theme and emotion, after which Google's AI tools will generate a soundtrack and sync it to the video.
“Video generation models are advancing at an incredible pace, but many current systems can only produce silent outputs. One of the next big steps in bringing generated movies to life is creating soundtracks for these silent videos,” DeepMind wrote in a blog post.
Below you can see an example of a horror score generated to go with a creepy video.
This has huge potential for creators, who may soon be able to use this tool to create completely unique soundtracks to perfectly match their content. It also opens the door to considering the reverse, where AI can automatically generate videos based on audio. This dramatically increases the possibilities for musicians to create ambitious, creative music videos, regardless of budget or equipment.
Google's V2A tool also provides the ability to automatically add voice narration to videos, potentially enhancing educational content by helping educators create engaging and interesting videos with a simple narration workflow.
V2A does more than just generate music. Its deep AI learning allows it to understand the content of your video and create a soundscape to match. They want to make it easy for creators to create content, and therefore the entire product.
But Google recognizes the importance of protecting creators in responsible AI development, and to ensure transparency, the company is integrating the SynthID tool, which marks content generated by V2A.
“To ensure that our V2A technology can positively impact the creative community, we are gathering diverse perspectives and insights from leading creators and filmmakers, and using this valuable feedback to inform our ongoing research and development,” DeepMind wrote.
They are testing the tool before releasing it to the public.
