Cologne-based translation company DeepL unveiled its new speech model in a press release on Wednesday 11 November. Aure produces the translated speech in DeepL Voice, which translates speech into another language in real time.

The Decoder and Trending Topics reported that DeepL Voice has produced translations in the speaker’s own voice since September. That voice, however, was based on a sample heard at the start of the conversation. DeepL said the fixed sample conveyed who was speaking but not how that person spoke. The translation did not follow changes in emotion, emphasis or pace, nor sarcasm, where the tone of voice contradicts the words.

Voice, prosody and content kept separate

Aure analyses speech throughout the conversation. The model decides how to deliver a sentence based on both the translated words and the original voice, the press release said. DeepL does not permanently store meeting transcripts or translations, so it has little data on the same speaker in different languages. The model must therefore adapt to the speaker mid-sentence.

In training, DeepL separated the speaker’s voice identity, prosody – the melody and rhythm of speech – and spoken content, so that each can be carried into the new language independently. The company used reinforcement learning to refine the model’s output so that translations stay accurate and speech sounds natural, without a foreign accent. The model also separates the speaker’s voice from background noise and echo.

South Korea’s Yonhap news agency wrote that Aure converts the audio directly rather than first transcribing speech into text. DeepL’s press release, however, said the model also uses the translated words, so the claim does not fully match the company’s own description.

Quality claims come from DeepL

The Decoder, heise online, Trending Topics and Yonhap base their reports on information from DeepL, and none of them presents an independent measurement of Aure’s quality.

Trending Topics cites a blind test commissioned by DeepL and conducted by the industry publication Slator. In it, 96 per cent of the linguists surveyed preferred DeepL Voice to the native translation tools from Google, Microsoft and Zoom. The test concerned DeepL Voice, and the source does not say whether Aure was involved. DeepL’s product page uses a different metric: it says meeting translations contain 76 per cent fewer critical translation errors than the platforms’ own tools.

Trending Topics reported that Google Meet’s translation and Microsoft Teams’ Interpreter agent also aim to mimic the speaker’s voice.

Availability and privacy

Aure is enabled automatically in DeepL Voice for meetings and face-to-face conversations, and in DeepL’s speech translation API. DeepL Voice can be tried free of charge, and the meetings version works in Microsoft Teams, Zoom and Google Meet, the product page says.

The initially supported languages are English, German, Spanish, French, Italian, Japanese, Portuguese and Korean. For other languages, DeepL Voice still uses earlier models. DeepL promises more languages.

Aure neither creates nor stores a persistent voice profile, the product page says. Voice data is processed in memory and deleted when the session ends, and DeepL does not use meeting data to train its models.