Google has embraced the AI era through the development of Gemini and other tools. The company has already shown off VideoPoet and Veo, which can generate videos from text input. The company's DeepMind AI division unveiled a new video-to-audio (V2A) technology to create contextual audio files for silent videos. Simply put, the technology can create dialogue and soundtracks for videos based on the scene. Here are the details:
Google's V2A technology explained
Google DeepMind's video-to-audio technology analyzes pixels in the video using natural text prompts to help the tool better understand the video content. Using this data and Google's in-house AI models, the V2A tool creates high-quality sound effects that match the video.
V2A is also using Google's video generation tool Veo to create realistic sound effects and try to match the tone of specific subjects in a video. The new tool can create audio, animation and stock footage content featuring people, and the company has posted some examples on its website that show the technology's potential.
Google's V2A Diffusion-based techniques, Like most multimedia-related AI tools, it uses a series of encoders and recorders combined with a trained diffusion model to create the final audio file. As with AI chatbots like Gemini and ChatGPT, training with additional data improves the tool's effectiveness.
V2A is currently a concept project and has not been released to the public. Google says that further research is ongoing to refine the V2A technology and achieve realistic results. Currently, there are several text-to-video and text-to-audio generators on the Internet, but Google's V2A is unique in that it can create audio for a user-provided video.
Google has yet to reveal a timeline for the public release of V2A, and Veo appears to be the company's priority project to rival OpenAI's Sora AI video generator.
