At the I/O developer conference, Google announced today that it is working on an experiment called universal translator. As its name suggests, this tool aims to translate videos from one language to another while preserving the overall tone and atmosphere. This means that this utility not only translates audio from one language to another, but also imitates the speaker’s voice, tone, and facial expressions. That’s right, the output video of a person speaking synchronizes lip movements based on utterances in the target language.
Given that this tool can be abused to create problematic deepfakes, Google restricts access to Universal Translator. As a result, only approved partners can use its power for constructive projects, excluding general users. While this move limits Universal Translator’s promising potential, it does help keep bad actors at bay.
As the graphic in the photo above reflects, Universal Translator first detects and translates words. Then check the tone of the speaker and what they are emphasizing. After combining these two aspects, we generate speech in the target language. Finally, we synchronize the lips of the speaker in the video based on the pronunciation of the AI-generated speech.
Apart from Universal Translator, Google today announced other AI-powered tools, including improved navigation in Maps and smarter Google Search. Notably, the company has also launched some of its much-anticipated hardware, including his Pixel 7a, Pixel Fold, and Pixel Tablet. However, given the current trend revolving around artificial intelligence, the company leveraged AI for a significant portion of its keynote rather than focusing primarily on hardware and upcoming Android OS features. Spent on previewing the tool.
Those interested in the main keynote will be able to rewatch the stream through Google’s official YouTube channel after it concludes.
