NVIDIA released the Nemotron 3.5 Bolt 74B-A9B 24GB model on Hugging Face on 15 October. Sam Witteveen, who tests AI models on YouTube, walked through how the model was made in a video published on 9 November.

Bolt is an open-weight mixture-of-experts (MoE) model: for each token it uses only some of its subnetworks, which are called experts. Of its 74 billion parameters, 9 billion are active per token. NVIDIA designed the model for AI agents that run locally and sized it to fit in the memory of the RTX Spark device.

Quantisation stores a model’s weights with fewer bits. Witteveen said rounding the weights of a finished model usually works down to four bits, but quality drops sharply between two and three bits.

Pruning first, quantisation last

According to the model card, NVIDIA pruned Bolt from the 120-billion-parameter Nemotron 3.5 Super VL model. An architecture search chose the number and width of experts layer by layer so that the quantised model fits in under 24 gigabytes with a 128,000-token context window. The number of experts used per token fell from 22 to nine.

The pruned model recovered its capabilities by learning from the parent model. NVIDIA then fine-tuned it and trained it with reinforcement learning. Finally, the company taught the quantised model to imitate its unquantised version. Witteveen said this is how the model learns to work at a low bit count.

NVIDIA stored all experts at 2.0625 bits per weight. The Mamba layers use the 4-bit NVFP4 format and the embedding and output layers are 8-bit, so the model is not entirely 2-bit.

NVIDIA cites earlier research, such as the ParetoQ and Quest preprints, to support the approach. According to these preprints, layers twice the size can perform better at two bits than the originals at four. Witteveen remains sceptical, because outsiders find it hard to replicate this kind of training.

The benchmark figures are NVIDIA’s own

In most of the 13 benchmarks on NVIDIA’s model card, the 2-bit Bolt scores higher than three smaller MoE models measured unquantised in BF16 format. The table shows five of the benchmarks.

Bolt does not come out on top in every benchmark in the table: Qwen3.6 is narrowly ahead in PinchBench, and some of the comparison models score higher on both AA-Omniscience metrics. According to NVIDIA figures presented by Witteveen, quantisation lowered the PinchBench score from about 90 to 88 and the GPQA Diamond score from about 89 to 86. Witteveen said the unquantised version takes up 148 gigabytes.

Witteveen: smarter, not faster

Witteveen said NVIDIA’s speed table puts Bolt at around 84 tokens per second on the RTX Spark, fewer than Qwen 3.6 and Gemma 4. Witteveen described Bolt as a model that is smarter per gigabyte of memory but not faster.

In Witteveen’s own test, Bolt scored 10/10 on basic tool-call tests. It sometimes failed to fend off prompt injection, as did Super. In Witteveen’s view, Bolt suits Spark-class devices, but on a more powerful GPU a larger model may be the better choice.

The model can be downloaded from Hugging Face under the OpenMDW-1.1 licence, which permits commercial use. A GGUF version is also available. The model card’s instructions use the development version of vLLM or newer. The model card lists the supported GPUs differently in different places: either A100, H100 and Blackwell GPUs, or only Blackwell’s SM120 and SM121 chips.