Why we built decision-machine-1

A support request arrives: “I cancelled last week and you charged me again.” Your application needs to choose a queue. The interesting question is what happens when the message concerns both billing and cancellation, or fits none of the queues you defined.

That is the problem we built decision-machine-1 around. Give engineers an explicit way to ask for a decision, inspect the result, and decide what their software should do next.

In his account of building milliseconds.ai, Baptiste Laget describes separating existing decision capabilities into their own service. His motivation was that short inference tasks needed a request path designed around their operating constraints.

Here is the product rationale behind that work.

The useful unit is a decision your application understands

A routing feature already has a vocabulary: billing, account access, cancellations. An extraction feature has fields. A triage feature has levels. The product defines these before a model gets involved.

We wanted that definition to be visible in the API. With classify, you supply labels and optional descriptions. The response includes the selected label and scores for the alternatives. With answer, you ask a question and receive a passage from the input, including its position, or a null result. These are different operations with different response contracts. The API specification makes those shapes explicit.

This gives a team something concrete to review. Are the categories mutually understandable? Does “cancellation” include a request to reverse a cancellation? Is an answer expected to appear in the supplied text?

Those questions remain engineering work. Putting them into named inputs makes them easier to find, test, and discuss. A label description deserves the same care as any other configuration that changes production behavior.

A narrow interface leaves room for different models

The name decision-machine-1 identifies the product surface developers call. It should not be read as a claim that every operation uses one newly trained foundation model.

Our Qwen-based runner uses a small instruction-following model for choices and GLiNER for spans. The choice path constrains generation to an option token and derives scores from the option log probabilities. The span path locates material in the source text.

That distinction matters. Qwen2.5-1.5B-Instruct is a causal language model. Using it for a constrained choice still involves an LLM and a generation step. GLiNER2.5 provides extraction capabilities suited to finding entities and structured information. An API that returns no free-form prose can contain both approaches.

The engineering objective is to fit inference to the operation. A constrained choice and a source span need different behavior. Keeping their public contracts explicit lets us evaluate the implementation behind each one without asking users to redesign every integration.

It also creates an obligation: changing a backend can change the decisions and scores. A stable JSON shape does not remove the need for behavioral regression tests.

Scores belong beside the answer

Suppose a classifier returns billing. That is enough to choose a branch, but it does not tell you whether cancellation was nearly as plausible.

For classification, decision-machine-1 returns the score distribution, the winning label’s normalized share, and a separate confidence value derived from the distribution’s entropy. The probability documentation explains those meanings per capability.

These numbers support a routing policy. They do not establish that a decision is correct. A model can strongly prefer the wrong category. If your label set excludes the right answer, a clear winner among the remaining labels is still a bad result.

We expose that information because the application owns the consequence. Moving a ticket between queues can tolerate a different error rate from authorizing a refund. Both may start with the same text interpretation; they need different rules before acting.

For a CTO, that separation is useful when assigning ownership. The model predicts. Product and engineering define acceptable mistakes, validate thresholds against real examples, and decide which cases need review.

Evidence should be inspectable

When an application asks for a date in a message, an answer is more useful if the interface can highlight where it came from.

The answer and entities endpoints return source spans with offsets. That creates a straightforward review experience: show the extracted text in context, let someone correct it, and retain that correction for evaluation. See the documented span contract.

A span provides evidence of what was selected. It does not prove that the selection answers the question. A message can contain several dates, several people, or a quoted statement that the writer rejects. Extraction makes a result inspectable; interpretation still needs testing.

This is why the product should make uncertainty manageable rather than hide it behind a polished answer.

Evaluate the boundary before expanding it

Start with one decision that already exists in your product. Write down its possible outcomes and the action each outcome permits. Then assemble representative inputs, including missing information, overlapping categories, and text that belongs nowhere.

Measure the whole feature: incorrect actions, review volume, request failures, and response time from the caller’s perspective. A useful evaluation also records which inputs were excluded and why. Otherwise, an apparently clean result can hide the cases your users encounter most often.

If the decision needs a conversation, extended reasoning, or generated prose, give it a system designed for that work. A small decision step can also precede a larger model, provided you test the escalation policy rather than assume that a score will separate every hard case. Our comparison with general-purpose LLM APIs examines that choice.

We built decision-machine-1 to make this part of an application explicit: the text, the decision being requested, the evidence returned, and the policy that acts on it. Pick one branch in your product and use the API documentation to test whether that contract fits.