ElevenLabs launches voice translation tool to break down language barriers for content

• Voice AI platform ElevenLabs unveils new AI Dubbing feature - the biggest milestone yet in the company’s efforts to remove language barriers and foster universal accessibility of content
• This voice translation tool can convert spoken content to another language in minutes, while preserving the voice of the original speaker. Built on top of the company's own research, this means audiences globally will be able to enjoy content in their native language
• The new AI dubbing feature empowers all creators, educators, and media companies to reach a global audience with a few clicks. This follows the recent launch of ElevenLabs’ Projects tool to support streamlined long-form audio creation

10th October, Global - ElevenLabs, the global leader in voice AI technology, today announces the addition of a groundbreaking voice translation feature to its platform. Born from the foundational mission to eliminate the linguistic barriers of content upon which ElevenLabs was established, and a culmination of the company's research to date, the AI Dubbing tool enables users to automatically translate any speech into a different language while maintaining the original speaker’s voice - creating a new, high-quality and effective way of dubbing audio and video content.

CEO and co-founder of ElevenLabs, Mati Staniszewski, comments:

“The release of AI Dubbing is our biggest step yet towards eliminating these linguistic barriers of content. It will help audiences enjoy any content they want, regardless of the language they speak. And it will mean content creators can easily and authentically access a far bigger audience across the world. Having experienced these language barriers first hand, my co-founder Piotr and I are so excited to finally bring AI dubbing to life.”

Signaling a new era in content accessibility across mediums like streaming, gaming, and films, the new tool goes beyond traditional translation and captioning by preserving the speaker's identity and original speech patterns when translated. This means that the voice of the same actor can be heard by audiences in over 20 different languages, delivering lines with the characteristics and nuance of the original recording.

AI Dubbing combines the company’s research on multilingual speech synthesis, voice cloning, text and audio processing, into a single tool; enhancing the authenticity of the translated content and making it more accessible to a broader audience while keeping the essence of the original delivery. The AI Dubbing feature supports voice translation across over 20 languages currently supported by the Eleven Multilingual v2 model, including Hindi, Portuguese, Spanish, Japanese, Ukrainian, Polish or Arabic.

This move further extends ElevenLab’s research across the audio AI space; enabling emotions and intonation from original audio to be transformed to dubbed audio. The company has also developed its own method to detect the voices of different speakers and methods to remove background noise, differentiating music and noise from dialogue. This means the tool can effectively handle audio or video containing multiple speakers, ensuring each voice remains distinct and recognizable across different languages.

By automating the dubbing process and ensuring high-quality output, ElevenLabs simplifies the creation of multilingual content for creators, educators, and media companies. Only 6% of the world’s population are native English speakers and 75% don’t speak any English. This release means every video can now be made available to a global audience, at speed and at low cost.

The announcement follows the recent release of Projects - ElevenLabs’ advanced workflow for generating and editing long-form audio, supporting a variety of editing features, such as speaker assignment, pause insertion, and segment regeneration.

With this addition, ElevenLabs continues its mission of breaking down language barriers and making content universally accessible in any language and voice.