DecisionBench and choosing a decision model

Leaderboards are starters, not finishers

DecisionBench-style leaderboards (community posts track pplx-decider checkpoints topping accuracy-per-cost charts) are useful priors. But your distribution is not the benchmark’s: your states, your criteria, your ambiguity profile.

Run your own

The Playground takes your state and questions and runs them against any model in the catalog, showing latency, tokens and per-run cost. Paste a sample of your real traffic, try the top three models, and let your own numbers pick.