Baidu releases ERNIE 5.5 Flash base model weights for further training
Written by AI
Baidu says its 280-billion-parameter MoE model beats base models of the same size in most of the company’s own tests. News sites report that the model’s licence does not allow use in the EU, the UK or South Korea.
On 5 November, Baidu’s ERNIE team published the weights of ERNIE-5.5-Flash-Base on Hugging Face as they stood at the end of pre-training. The model has not been fine-tuned or taught to hold conversations. Baidu offers it as a starting point for users’ own fine-tuning and for reinforcement learning research.
Because it is a base model, it is used through a text-completion interface rather than a chat interface. Baidu plans to release a post-trained version later, The Decoder and Pandaily reported.
How the model saves compute
The model has 280 billion parameters, but 14 billion of them process each token. The saving comes from the mixture-of-experts (MoE) design. Each of the model’s 42 MoE layers has 512 specialised sub-networks, or experts, and the model selects ten of them for each token. One shared expert also handles every token. In Baidu’s design, several routers make the selection instead of one.
The model caches information about earlier tokens, which takes up a lot of memory in long texts. Of the model’s 43 layers, 37 store one entry per 128 tokens, and six keep access to every token in the full context.
Baidu pre-trained the model on 33.5 trillion tokens with a context of up to 128,000 tokens. The company says the context window can reach 1 million tokens in fine-tuning and inference, although the model was not trained on contexts that long.
Baidu’s own comparisons
Baidu compared the model with DeepSeek-V4-Flash-Base, Tencent’s Hy3-preview-Base and Xiaomi’s MiMo-V2.5-Base. The company tested all the models with the same evaluation software and the same prompts. By MarkTechPost’s count, ERNIE scored best in 24 of Baidu’s 30 benchmarks. The table shows a selection of both wins and losses.
| Benchmark | Metric | ERNIE 5.5 Flash Base | DeepSeek-V4-Flash-Base | Hy3-preview-Base | MiMo-V2.5-Base |
|---|---|---|---|---|---|
| MMLU-Pro | EM | 78.4 | 69.2 | 66.9 | 65.7 |
| SimpleQA | LLM judge | 34.1 | 29.1 | 25.9 | 20.8 |
| GPQA | EM | 60.2 | 55.9 | 50.2 | 51.4 |
| HumanEval+ | pass@1 | 82.3 | 69.5 | 78.0 | 74.4 |
| LiveCodeBench | pass@1 | 40.1 | 31.9 | 33.5 | 32.4 |
| LongBench-v2 | EM | 48.7 | 44.5 | 39.8 | 44.6 |
| ChineseSimpleQA | LLM judge | 73.0 | 75.3 | 73.2 | 68.8 |
| BBH, chain-of-thought | EM | 89.2 | 87.5 | 90.0 | 89.8 |
| FullStackBench | pass@1 | 54.6 | 43.3 | 56.7 | 54.1 |
| HELMET-Rerank | NDCG@10 | 56.2 | 60.0 | 55.7 | 56.6 |
| InfiniteBench | average | 46.5 | 48.0 | 33.0 | 46.7 |
Source: Baidu’s model card, the company’s own measurements. EM is the share of exactly correct answers; pass@1 is the share of code that passes the tests on the first attempt.
The results are self-reported by Baidu, and the metrics vary between benchmarks. MarkTechPost noted that Baidu’s figures for competitors differ slightly from the developers’ own. DeepSeek, for example, lists an MMLU-Pro score of 68.3, whereas Baidu measured 69.2.
Licence restricts use
The Decoder calls the model an open-source base model, but the weights are covered by Baidu’s own licence. Unite.AI and MarkTechPost reported that the licence territory does not include the EU, the UK or South Korea. In those regions the model may not be used, modified or distributed, Unite.AI wrote.
Unite.AI’s article was written by AI and checked by its editors. It says non-commercial use is free. If products using the model earn annual revenue of up to $50 million, commercial use requires a one-time notice and a visible ‘Powered by ERNIE’ attribution. Above that, Baidu’s prior written permission is required. The licence prohibits using the model or its outputs to improve AI systems unrelated to ERNIE.
The weights are available on Hugging Face and ModelScope in FP8 and BF16 formats, and the model card recommends running the model with SGLang or vLLM. Baidu does not specify the required GPU model or amount of memory, MarkTechPost reported. Unite.AI wrote that NVIDIA’s guide uses 16 servers, each with four GB300 GPUs, to fine-tune the full model.
Read more: Models