DeepL’s New A.I. Model Aure Carries Emotion Across Languages in Real Time

Set Trending Topics as a preferred source on Google.

First your own voice, now the mood behind it: DeepL has just unveiled Aure, a new voice A.I. model that carries a speaker’s tone, emotion, emphasis and pace into real-time speech translation.

Anyone who suddenly sounds excited, serious or ironic in a meeting should come across the same way in English, Japanese or Korean. According to the announcement, Aure is the technology behind DeepL Voice, the real-time translation product of the Cologne-based A.I. unicorn, and the company’s most expressive voice model to date.

From a Snapshot to Constant Listening

In September, DeepL introduced a feature that translates users in their own recognizable voice. That system had a weakness, however: how the translated voice should sound was determined from the first moments of a conversation. The model knew who was speaking, but any changes later in the conversation were lost. According to DeepL, such a fixed snapshot can neither reflect shifts in mood nor capture delivery that contradicts the words being spoken, such as sarcasm.

Aure therefore analyzes incoming speech continuously, using both the source audio and the words already translated. From this, the model infers how a sentence should be delivered. Someone who pauses before an important announcement, speaks with urgency in a negotiation or shifts from serious to excited while sharing good news should have those nuances come through in the translation. Even a person’s mood on a given day plays a role: someone may sound tired one day, enthusiastic the next or different under stress. Aure is designed to recognize these variations and adapt the translated voice to the moment.

Jarek Kutylowski, DeepL’s founder and chief executive, said our voice is fundamental to who we are and to how we express emotion, collaborate and connect with others. For too long, he added, communicating across languages has meant compromising on how people express themselves. With Aure, Mr. Kutylowski said, even the subtlest changes in expression can be captured and carried into another language, in the speaker’s own voice and in real time.

How DeepL Solved It

According to DeepL, three hurdles stood in the way. First, real conversations are rarely quiet: Aure has to isolate a speaker’s vocal identity and prosody while filtering out background noise, echoes and distortion. Second, reproducing a voice across languages is far harder than within one language, especially without exaggerating accents or distorting how a person sounds.

Third, there is a shortage of data. Recordings of the same person speaking several languages are rare. On top of that, DeepL Voice does not permanently store meeting transcripts or translations and deletes them from memory once a session ends. Aure therefore has to adapt to each speaker live, before they have even finished a sentence.

DeepL addressed this by combining several training methods that separate vocal identity, prosody and spoken content, so that each layer can be transferred into the new language independently. Reinforcement learning then refines the output to preserve translation accuracy and audio quality and to produce natural, accent-free speech.

Eight Languages at Launch

Aure is applied automatically, with no extra configuration, in DeepL Voice for online meetings, DeepL Voice for in-person conversations and the DeepL API for voice. At launch, the model supports English, German, Spanish, French, Italian, Japanese, Portuguese and Korean, with more languages to follow. All other languages continue to run on DeepL’s existing voice models.

According to the company, DeepL Voice customers include Pioneer Corporation, Aramark, Avendra International and Inetum. Spoken translation has become DeepL’s most important growth area: In April, the company launched voice-to-voice translation, and in May Mr. Kutylowski announced that about 250 jobs, roughly a quarter of the work force, would be cut as DeepL sharpened its focus on real-time speech translation.

Competition From Big Tech

DeepL is not alone in chasing emotional A.I. voices. The large platforms that DeepL Voice plugs into are building similar features themselves. Google Meet, for example, offers real-time speech translation powered by Gemini that aims to reproduce a speaker’s tone and voice. Microsoft Teams has its own A.I. interpreter in the Interpreter agent, which can simulate the speaker’s voice.

DeepL is competing primarily on quality. In blind tests commissioned by DeepL and conducted by the industry publication Slator, 96 percent of the linguists surveyed preferred DeepL Voice over the native translation tools from Google, Microsoft and Zoom.