Hugging Face described decision models in a blog post on 10 November. A decision model takes a description of a situation, such as a customer message, together with named questions and their answer options. Instead of writing an answer token by token, the model assigns a probability to each permitted option.

Three question types

There are three question types. With choice, the model picks one option and returns the full probability distribution. With score, the model rates on an ordered scale and returns the expected value of the levels’ ordinal positions. With noul, the model returns the probability that the answer to a yes–no question is yes. The application sets the threshold.

In the blog’s example, the autotrust/GEV-26B-Decide model routed a refund request for a double charge to the billing team with a probability of 0.998. The model also estimated, with a probability of 0.995, that the customer was asking for a refund. Urgency on a low–medium–high scale (0–2) scored 1.71. Hugging Face said the figure is not a level chosen between medium and high but the expected value of the model’s distribution. For choice and score questions, the model also returns a confidence figure: 0 when all options are equally likely and 1 when all the probability sits on one option.

Hugging Face justifies the model type by noting that a large language model will not necessarily follow the requested JSON structure. Constrained decoding enforces the structure by allowing only tokens that fit it, but it adds work. Outlines and XGrammar, for example, offer it.

Hugging Face does not claim that decision models invented runtime-defined labels or probabilistic classification. Zero-shot classification already lets users define the options at runtime. Decision models differ, the company said, because they have a consistent interface and one request can hold different question types. The blog mentions calibrated decisions only as a possibility that specialised training may bring. A calibrated model’s probabilities match actual hit rates. NewMediaFactor, summarising the blog, nevertheless describes the probabilities as calibrated.

A common interface for different implementations

Hugging Face aims for a common interface but does not force models into a common implementation. The Transformers library now has the DecisionPipeline, AutoModelForDecision and DecisionConfig classes, plus the /v1/systemone server API. In the example response from GEV-26B-Decide, the model generated zero tokens. llama.cpp also supports the same API.

The interface also works with compatible ordinary language models. Transformers builds the prompts, scores the answer options in a batched forward pass and converts the scores into probabilities. Users can therefore try the decision-model approach without training a new model, Hugging Face said.

Decision models have their own tag on the Hub. They can be based on a ModernBERT encoder or on large generative language models, for example. Hugging Face describes the field as young and fast-changing.

Decision models can be fine-tuned with the Transformers Trainer API. Support in the TRL library is still pending in an open pull request. Unsloth’s guide shows how to turn a generative language model, such as Qwen, into a decision model with LoRA fine-tuning.

Using decision models requires a Transformers version that supports them, and a local server needs the library’s server dependencies. Instructions and code examples are on Hugging Face’s blog.