Udgam Labs said in a blog post on 7 November that its Shruti speech-to-text models now cover all 22 languages in the Eighth Schedule of India’s Constitution. Each language has its own model, and every model also recognises English.

The models are built on Cactus Whistle, a speech model from Cactus Compute. It is an open-weight model with 55 million parameters. Udgam Labs added vocabulary for each language and fine-tuned all the model weights on public speech datasets. The weights are quantised to 4 bits, so each model file is 29.5–29.7 MB.

The models write speech in each language’s own script. They often leave English words in Latin script, as many people do when they write: ‘मेरा order कब आएगा’.

Gains depend on how errors are scored

The company measured accuracy by word error rate, the share of words recognised incorrectly, so a lower figure is better. The main test was Indic DiarBench from Sarvam AI and AI4Bharat. Its real conversations were not in any model’s training data, the company said.

Udgam Labs said 11 Shruti models make fewer errors than IndicConformer for the same language when an English word counts as correct only if written in Latin script. In Gujarati, for example, Shruti scored 28.5, SraVaani-1.0 32.4 and IndicConformer 44.9.

If English words must be written in the language’s own script, IndicConformer or SraVaani-1.0 is more accurate in most languages. On read speech, the larger models are clearly more accurate in most languages. On the FLEURS test, for example, IndicConformer Punjabi scored 16.9 and Shruti Punjabi 29.3. In conversations, between a quarter and more than half of words are recognised incorrectly, depending on the language.

Some tests are small: in the conversation test, 11 languages have fewer than 100 speaker turns. For seven languages, the company presents no comparison it considers reliable. All figures are Udgam Labs’ own measurements.

The Hindi model file is about 100 times smaller than OpenAI’s Whisper large-v3. In the company’s Hindi tests it made fewer errors; on the Monsoon test, for example, the scores were 20.46 and 33.29. The company ran Whisper without its temperature fallback, which caused repetition loops. Shruti was more accurate even when clips stuck in repetition loops were removed, the company said. In English, Whisper is clearly better. Hindi-only models about 16 times larger are much more accurate, according to the company.

Limitations and deployment

The company has not tested the newer models in noise or with music, nor most languages with microphone input. In heavy noise, the Hindi models sometimes repeated the same phrase. Native speakers have not checked the transcripts for the 16 newer languages.

In the company’s measurements, 21 newer models ran on a five-year-old phone with 6 GB of memory. They transcribed 8.6–12.6 times faster than real time and used 64 MB of memory.

ONNX packages are also available, for example for browsers and Android. None of the packages passed all six of the company’s comparison rules. Nine failed the test, but the company released them anyway and disclosed the deviations. In some cases, a deviation meant fewer errors than the model running on Cactus’s needle engine.

The models are on Hugging Face under the Apache-2.0 licence, and runnable examples are in the company’s GitHub repository. The licence does not set terms for the training data, and no licence is named for two datasets used for Shruti Assamese. The models run on the needle engine without a key or account, but the first download requires an internet connection unless the model ships with the app. By default, the engine’s Python package sends anonymous usage statistics; users can switch this off with the environment variable NEEDLE_TELEMETRY=0.