This new AI dubbing tool can translate an audio document while preserving the speaker’s voice

When you watch a dubbed movie or show, you know how unnatural the sound you hear is. Often, the speaker’s voice does not change from one character to another, or the tones and fluctuations of dialogue are completely wrong. ElevenLab’s new AI feature aims to remedy this problem.

ElevenLabs has indeed recently unveiled AI Dubbing, a new feature that allows you to convert spoken content into another language while preserving the voice, speech patterns, emotions and intonations of the original speaker.

The AI Dubbing function aims to allow audiences around the world to enjoy their favorite content in their native language by converting it in just a few minutes.

Combination of speech synthesis, voice cloning,
text and audio processing

The AI Dubbing function combines ElevenLab’s multilingual speech synthesis, voice cloning, text and audio processing to preserve the nature of the original content when creating a translation, according to the company.

In the press release, ElevenLabs CEO and co-founder Mati Staniszewski shared his negative experience with dubbed content as a child and explained how the new technology could be a solution. “Growing up in Poland, we watched English-language films that had been dubbed by a single narrator. This means that all the actors have the same voice. It takes a lot of magic out of the experience,” Mr Staniszewski said. “The release of AI Dubbing is our biggest step forward towards eliminating these content language barriers. It will allow the public to enjoy all the content they want, regardless of the language they speak.”

The AI Dubbing function will support more than 20 languages currently supported by the Eleven Multilingual v2 model.

French in the list

The full list of languages includes English, Japanese, Chinese, german, hindi, french, korean, portuguese, italian, spanish, Indonesian, dutch, turkish, Filipino, polish, swedish, Bulgarian, romanian, Arabic, Czech, greek, Finnish, Croatian, malay, slovak, Danish, Tamil and Ukrainian. AI dubbing is available to all users from today, and the price depends on the number of characters you want to double.

Source: “ZDNet.com “