System 1 vs System 2 models: decision models vs generative LLMs

Two kinds of compute

System 2 LLMs generate token by token — flexible, expensive per output, latency in seconds. System One (decision) models evaluate state against typed questions in a single forward pass — milliseconds, structured answers, no prose.

The categories are complementary: generate with System 2, decide with System One. Intent routing, moderation gates, grading, permission checks and evaluation verdicts are decisions wearing an LLM costume.

Why output is free

A decision is a handful of typed values, not a stream of tokens — so catalogs price input only. That is also why decision models pair naturally with high-volume pipelines: cost scales with what you need to evaluate, not with what needs saying.