Reasoning models buy accuracy with time — pay only where errors are expensive
Reasoning models & test-time compute
The question: What actually changed with "thinking" models, and when is the slower, costlier call worth it?
The o-series/R1/extended-thinking generation added a new dial: models that spend more compute AT ANSWER TIME to reason step-by-step. It reshapes cost curves, routing decisions, and what "hard" means for a task.
What the lesson covers
The shift: pre-2024 scaling improved models by training bigger; test-time compute improves answers by letting the model think longer on YOUR question — exploring steps, checking work, backtracking. Accuracy on maths, code and multi-step logic jumps; latency and price jump with it. It is a per-question dial, not a blanket upgrade.
Where thinking pays: multi-step quantitative work, tricky code, constraint puzzles, legal/logical chains where one slip invalidates the answer — domains where verification is easy but generation is hard. Where it does not: lookup, summarisation, style transformation, most chat — the standard model is faster, cheaper and just as right.
The routing consequence extends Lesson 8: your cost strategy now has three lanes — small model for routine, frontier for nuanced generation, reasoning lane for verified hard steps. Mature systems route per task and CHECK the reasoning lane's work with cheap validators (tests for code, arithmetic checks) because thinking reduces, not eliminates, error.
Keep the claims honest (Lesson 4's jaggedness applies): reasoning models still hallucinate confidently outside their strengths, their visible "thoughts" are useful but not a faithful window into computation, and benchmark gains do not automatically transfer to your task. The evaluation discipline is unchanged: your data, your metric, your baseline.
The budgeting question this creates is new, and it is worth stating plainly because it does not resemble any earlier cost line. Reasoning models turn accuracy into a dial you pay for per question, which means the right amount of thinking is a business decision made per workflow rather than a setting chosen once. A step where an error costs a thousand euros and verification is cheap deserves the expensive lane; a step where an error costs a retry deserves the cheap one. Most organisations discover instead that they have routed everything to one lane — usually the expensive one, because it demos better — and the fix is not a cheaper model but a routing rule with the error cost written into it.
Key points
- Test-time compute = paying for thinking per question; accuracy up on multi-step logic, cost and latency up too.
- Pays where generation is hard but verification is easy (maths, code, chains); wasted on lookup and style.
- Three-lane routing: small · frontier · reasoning — with validators on the reasoning lane.
- Jaggedness persists: visible thoughts ≠ faithful computation; evaluate on your task.
- Reasoning turns accuracy into a **dial you pay for per question**, so the right amount of thinking is a per-workflow decision. Route on the cost of an error, not on which model demos best.
Framework — Three-lane routing
Small/fast for routine · frontier for nuance · reasoning lane for verified hard steps — each with its own cost, latency and validation. Route per task, not per vendor.
The lab
Find where thinking pays in YOUR work — empirically.
Open this lesson, its lab and its quiz
Sources and further reading
- AI Index — technical performance & reasoning — Stanford HAI
- Extended thinking — docs — Anthropic