The llama.cpp project released version b12180 on GitHub on 6 November. llama.cpp is an open-source program for running language models on your own hardware. Its converter drops Kimi-K3.1-Pro’s vision tower and mm_projector, so the model is text-only for now, the release notes say.

The change description says K3.1 builds on its predecessor, Kimi K3, but replaces the full-attention MLA layers with sparse-attention DSA layers. In sparse attention, a lightning indexer scores earlier tokens, and the model computes attention only for the highest-scoring ones.

On this checkpoint, 24 of the model’s 93 layers carry the indexer, which selects 2,048 tokens. Everything else is unchanged, including the linear-attention KDA layers. llama.cpp therefore reuses K3’s computation and adds the indexer.

The indexer’s selection is applied as a mask over the whole cache. The selected rows are not gathered separately. This lets the model reuse llama.cpp’s existing sparse flash attention path. Off that path, the result stays correct, but attention is still computed over the whole cache.

In a mixture-of-experts (MoE) model, each token is processed by only some of the model’s subnetworks, known as experts. K3.1’s routed experts stay in the same 4-bit MXFP4 format as in K3 and are not dequantised to a higher-precision format. The indexer’s three tensors are kept out of the per-tensor quantisation overrides, because token selection is sensitive to their precision.

The author of the change tested support on an 8-layer slice of the model, not the full model, against the model’s Hugging Face checkpoint. The author reported that 38 tensors matched exactly and that the indexer metadata matched the model’s config.

The selection was also compared with full attention using perplexity, which measures how well a model predicts text. When the number of tokens to select exceeded the context length, the result matched full attention. When the number was forced below the context length, the results diverged.

Prebuilt binaries for macOS, Linux, Windows and Android are available on GitHub, including CPU, CUDA, Vulkan and ROCm builds. Users must first convert the model’s Hugging Face weights with llama.cpp’s converter.