Deepdub GO uses a generative AI platform to professionally dub any video content

Language localization startup Deepdub Ltd. today announced the launch of a new dubbing tool powered by generative artificial intelligence that allows anyone to dub video content into 65 different languages.

This new tool is called Deepdub GO and uses similar generative AI technology that powers OpenAI LP’s famous ChatGPT, albeit for a very different use case. It’s a more accessible version of Deepdub’s proprietary AI platform for Hollywood studios, used to dub TV shows and movies in a variety of languages.

As Deepdub explains, dubbing has long been used in the entertainment industry to create a more engaging and immersive experience for viewers who don’t want to rely on subtitles. While this makes the content more accessible to a wider audience, due to the complexity and cost of the process, dubbing has generally only been used by film studios and video streaming platforms such as Netflix.

Deepdub GO makes dubbing possible for everyone, founder and CEO Ofir Krakowski told SiliconANGLE. He explained that he and his brother and co-founder Nir spent months looking at different potential use cases for generative AI and exploring ways the technology could have a positive impact.

“We realized that content was inaccessible to billions of people around the world because it was always created in a specific language and culture,” Krakowski said. “We are both big fans of the magic of great storytelling and content creation, so we found dubbing to be a perfect fit. ”

Deepdub’s mission is therefore to use generative AI to make content and knowledge more widely available through dubbing. To dub any kind of content, users simply upload their videos to the Deepdub GO platform, select the video style and preferred language, and the platform does the rest. Once the first dub is complete, users can edit and adjust it using various tools to make sure it sounds right.

“Deepdub GO is a web-based professional platform that elegantly encapsulates many of the same cutting-edge generative AI technologies we use for film and TV dubbing,” explained Krakowski. “Its user-friendly interface enables professionals and novices alike to create and localize audiovisual content with vivid, natural-sounding voices in dozens of languages. It was designed for , so it has best-in-class security measures built in.”

According to Krakowski, Deepdub GO allows users to translate and dub all kinds of content, including fitness videos, educational content, and even TikTok posts. Features include realistic, human-like voices, voice-guide technology that encourages emotions, and allows users to edit dubbed audio.

The emoting feature is particularly good, giving editors fine-grained control over the emoting of the dubbed audio. According to Krakowski, it uses customized large-scale language models to control speech performance through text-based prompts.

“It’s based on writing and language comprehension,” he explained. “The basic voice style that is created includes the emotion associated with this kind of text, which is similar to human speech. , giggle, whisper, shout, laugh, or change to a different style of emotion that suits your needs.”



Audio guides, on the other hand, are a technology that achieves theater-grade sound quality. According to Krakowski, users provide audio recordings that serve as guides for dubbed voice emotes, and Deepdub GO replicates them in the target language and voice.

The platform also supports voice cloning, allowing you to translate the original voice into foreign languages ​​such as Portuguese, Spanish, German, French, and Japanese. Otherwise, users can choose from thousands of artificial voices. Extensive editing features also allow you to change the translation itself, voice characteristics, translated voice length and even word count for more accurate lip-syncing.

Andy Thurai, an analyst at Constellation Research Inc., said that dubbing is one of the coolest and most useful applications of generative AI because it can translate video content with minimal effort. Applications include TV commercials, online learning, virtual human presentations, and real-time translation of speeches by business executives and politicians, he said.

“Most companies have historically avoided localizing video and audio content because it is very expensive,” Turai said. “Deep dubs are very effective. For audio rewrites, we take the complete script and rewrite it in another language. It also handles the added complexity of lip-syncing. It also helps with peeping, masking, changing to another language on the fly, and live translation as it happens.”

Thurai pointed out that DeepDub Go’s evocative and emotional resonance features help reduce the potential for emotionless, machine-like voices, a major problem with dubbing software. However, he noted that DeepDub is not the only AI-powered dubbing platform in the city.

“There are many alternatives out there,” he said. “One of my favorites is a company called Flawless AI that offers a unique and very capable solution. In addition, Google Translate offers similar functionality for free. DeepDub Go’s main differentiator is It supports 65 languages, which appeals to a lot of people.”

Deepdub GO is now available as a standalone platform as well as an application programming interface, allowing you to integrate it into your existing video editing tools and workflows. Free, Basic and Professional plans are provided to meet the needs of different users.

“Deepdub GO enables global storytelling, empowering content creators to resonate with their audience in a deeper and more personal way,” added Krakowski. “Our studio his platform allows creators to extend the potential reach of any content beyond traditional geographic and linguistic boundaries.”

Image: Deep Dub