Salesforce’s SOV-Transcribe-1 is the most accurate open-weight model in AA’s transcription test
Written by AI
In Artificial Analysis’s measurement, Salesforce’s SOV-Transcribe-1 transcribed 2.3 per cent of words incorrectly, the best result among the open-weight models AA compared. The weights can be downloaded to your own hardware under the Apache 2.0 licence.
On 12 November Salesforce released SOV-Transcribe-1, a model that turns audio recordings into text, on Hugging Face. Artificial Analysis (AA), which compares models, published its own measurement on X the same day. The AA-WER Index measures the share of words a model transcribes incorrectly, and a lower figure is better. The model scored 2.3 per cent. AA said this makes it the best open-weight model and seventh of 44 models in a comparison that also includes closed commercial models.
In the chart AA published, the next-best open-weight models both score 2.8 per cent: Mistral’s Voxtral Small and NVIDIA’s Nemotron 3.5 Transcribe. Alibaba’s Fun-Realtime-ASR-preview leads the whole comparison at 1.7 per cent.
In a second post, AA said the model transcribes a minute of audio in about a second on Salesforce’s hosted service.
Architecture and parameters
The model processes the whole recording at once rather than producing text as the speech progresses. The Chinese site hub.baai.ac.cn (智源社区) published an article, compiled by the AI agent AREX, that summarises the model card. That article says the model combines a Whisper large-v3 audio encoder with the Gemma 4 26B A4B language model. The language model is a mixture-of-experts (MoE) model, which uses only some of its parameters to process each token. All the parameters must still be held in memory.
The sources give different parameter counts. Salesforce’s blog puts the active parameters at 4 billion and the total at 26 billion. AA gives 4.6 billion active parameters. The model card lists 4.6 billion active and 26.6 billion in total, including the audio encoder.
The company’s own comparisons
AA has published the model’s word error rate and speed. The other results are Salesforce’s own. The company said the model formats text according to instructions written in plain language. When given keywords such as rare names, the model transcribed them more accurately in the company’s test: the keyword F1 score rose from 71.9 to 96.8.
The table shows the word error rate Salesforce reported in four tests. The company chose both the tests and the models it compared against.
| Test (word error rate, %) | SOV-Transcribe-1 | Nemotron 3.5 Transcribe | Voxtral Small |
|---|---|---|---|
| AA-WER Index | 2.28 | 2.81 | 2.77 |
| HALAS (hallucination-prone recordings) | 25.69 | 31.59 | 27.35 |
| Open ASR, multilingual average | 4.88 | 3.87 | 3.39 |
| ContextASR-Bench without context | 7.03 | 3.34 | – |
Source: Salesforce’s release. Open ASR and ContextASR rows from AREX’s article. Lower is better.
AREX reported that in the Open ASR test the model trails both rivals in all six languages other than English. Finnish is not among the supported languages listed in the model card. These are English, French, German, Spanish, Italian, Portuguese and Dutch. The longest recording tested lasted 30 minutes.
The weights can be downloaded from Hugging Face under the Apache 2.0 licence. The model card says a single RTX PRO 6000 Blackwell Server Edition GPU, paired with a helper model (SOV-Transcribe-1-DFlash), transcribes up to 1,039 seconds of audio per second. The figure was measured with a November 2026 development version of vLLM that has support for the model built in. Official vLLM does not yet include this support, which has only just been submitted to the vLLM developers. For now, the model runs through a plugin. With the helper model, the limit on concurrent requests must be lowered to 64, because vLLM’s default of 1,024 exhausts the memory and the model does not fit in 96 gigabytes of GPU memory. Salesforce said it will publish a technical report later.
Read more: Models