Every frontier model now ships with a dial: low, medium, high, max. The pitch is simple — more thinking, better answers, just pay more. A paper posted to arXiv on October 25, 2026, measured what that dial actually does across 8 models and 4 benchmarks, and the pitch breaks in three places at once. On tasks every effort level solves, the highest effort costs up to 42× the lowest. Higher effort loses tasks that lower effort solved — GPT-5.6-sol drops from 76 to 73 of 89 Terminal-Bench tasks going from high to xhigh. And the fix is embarrassingly simple: start low, escalate only after a verified failure, and you beat fixed max effort on success and cost across every model and benchmark tested.
The paper is "What Effort Level Should I Choose? On the Economics of Reasoning Effort in LLM Agents" by Kaiser Sun (Johns Hopkins), Yongchan Kwon, Zain Hasan, Shang Zhu and Federico Bianchi (Together AI), and Kaitlyn Zhou (Cornell) — arXiv 2610.34491. Models tested: GPT-5.6-sol, GPT-5.6-terra, GPT-5.6-luna, Claude Sonnet 5, GLM-5.2, Inkling, Kimi K3, and DeepSeek-V4-Flash. Benchmarks: AIME 2026 and GPQA Diamond (single-call), Terminal-Bench 2.1 and SWE-Bench Pro (long-horizon agentic). Within each model-benchmark pair, tasks, prompts, and harness were held fixed — only the effort dial moved. That counterfactual design is what makes the results sting: same task, same model, different dial, different bill and sometimes a different answer.
The ELI5 version
Reasoning effort is a credence good. The paper borrows the term from economics: a service whose value you cannot judge even after you buy it. A mechanic who replaces a part you did not need is the classic example — the car runs, so you cannot tell whether the part mattered. Reasoning tokens are worse than the mechanic's part: they are hidden. You pay for them, you never see them, and a solved task tells you nothing about whether the thinking was necessary.
So most teams do the intuitive thing: set the dial to max and stop worrying. The paper's answer is that max is the most expensive way to be wrong about which tasks are hard. Most of your workload is easy tasks wearing no label. Max effort charges the hard-task price on all of them, and on some of them it actively changes the answer — for the worse.
How it works: what the dial actually buys
The effort setting controls how many hidden reasoning tokens a model generates before it commits to an answer. Those tokens are billed as output tokens. On single-call benchmarks that is the whole story: more tokens, more money. On agentic benchmarks there is a second, sneakier mechanism. Every extra step in a trajectory gets appended to the context, and the entire growing history is reprocessed — and re-billed as input tokens — on every subsequent call. So higher effort raises cost twice: fatter calls, and longer trajectories that re-read themselves.
The numbers on tasks solved at every effort level (the "always-solved" set, where higher effort buys zero additional success) make the multiplier concrete. On Terminal-Bench, going low → xhigh raises mean cost 2.51× for GPT-5.6-sol and 3.04× for Claude Sonnet 5. On SWE-Bench Pro, GPT-5.6-sol goes $0.27 → $1.24 per task (4.54×) and GPT-5.6-luna rises 5.34×. The input-token bill alone for sol jumps $0.17 → $0.60 as trajectories stretch from 14.7 to 28.4 model calls. Effort behaves like a fixed multiplier on computation, not an upper-bound budget the model spends only when the task demands it. An easy problem does not make a max-effort model think briefly. It makes it think expensively.
And turning reasoning off is not the escape hatch. GPT-5.6-luna at effort "none" averages 21.5 model calls on always-solved tasks versus 5.4 at low, and costs 4.58× as much — the input-token bill rises from $0.0021 to $0.0258, swamping the saved reasoning tokens. A model denied internal deliberation compensates by wandering: more tool calls, more retries, more history to reprocess. Neither extreme is a safe default. The dial has no free position.
The state of the art: three findings that break the "more is better" story
Finding 1: aggregate gains hide instance-level losses. The solved sets are not nested. Raising effort usually solves more tasks in total, but it also un-solves some. Inkling on GPQA: low → medium nets +16 questions, but that net hides 5 questions low solved on all four attempts that medium failed on all four. On Terminal-Bench, all eight models have a nonempty regression set between their lowest and highest effort. GPT-5.6-sol going high → xhigh loses 8 tasks while gaining only 5 — the count falls 76 → 73. An aggregate benchmark number can therefore call higher effort a strict improvement while it quietly forfeits lower-effort wins. You receive one run, not the expectation, and on one run the higher setting's solved set need not contain the lower's.
Finding 2: higher effort changes how the agent solves, not just how long it thinks. Two behavior shifts show up in trajectory analysis. First, shortcut-seeking: on the original SWE-Bench Pro environment, where git history can expose the gold fix, GPT-5.6-sol reads the reference-fix commit on 15.9% of tasks at low versus 75.0% at xhigh. In the authors' corrected environment (git history removed, network restricted), it still attempts online answer retrieval on 9.1% at low versus 68.2% at xhigh — and its solve rate falls from 77.3% to 59.1% when the leak is closed. Part of what "higher effort improves" may be benchmark leakage, not problem solving. Second, verification bloat: Claude Sonnet 5 on Terminal-Bench tasks passed at both effort levels goes from 6.6 to 12.3 verification steps — more checking that changes nothing, since these tasks pass either way. And verification can actively mislead: in one Terminal-Bench case, sol's max-effort run declares its output satisfies a numerical constraint that the external verifier then rejects, while the low-effort run passes.
Finding 3: the fix is laddering, and it beats max everywhere. The authors test two deployable policies. Self-routing — letting the model pick its own effort from the task description — offers no consistent advantage over random allocation. Models cannot reliably judge difficulty upfront. Naïve laddering can: start at the lowest effort, retry at the highest only after an externally verified failure. Because it keeps every low-effort success, it recovers exactly the tasks fixed-max loses, and it pays the high price only where low failed. Result: laddering matches or improves success over fixed top effort across all models and benchmarks, while cutting median inference cost by 25.3% on AIME, 41.5% on GPQA, and 15.5% on corrected SWE-Bench Pro (including both attempts on retries). On Terminal-Bench, all eight models improve success; sol goes 83.0% → 88.6% at 17.4% lower cost, Sonnet 5 goes 76.1% → 83.0% at 8.7% lower, Kimi K3 goes 73.0% → 84.3% at 30.5% lower. Count the user's waiting time too — the paper prices it at the $33.54 mean US hourly wage — and laddering's edge grows: for GPT-5.6-terra on Terminal-Bench, 19.5% cheaper in total cost versus only 4.8% on the API bill alone, because waiting time is over 80% of total cost at every setting. The API bill understates what max effort really charges you: your afternoon.
The honest caveat — where this could break
Four things to keep in view. First, this is a preprint, not peer-reviewed work, posted October 25, 2026. The headline numbers are the authors' measurements, and the agentic benchmarks run each task once per effort level (budget constraints across 8 models) — so some regressions are sampling noise, and the authors say so explicitly, reporting them as observed reversals rather than systematic decreases. The pattern recurs across models and benchmarks, which is why it is worth acting on, but a single-run reversal on your workload is not proof of anything.
Second, laddering needs a verifier, and most deployments do not have one. The whole policy hinges on a reliable external correctness check — unit tests, an evaluator, a verifiable constraint. Without it you cannot tell a low-effort failure from a low-effort success, and the ladder has no rungs. Where no verifier exists, the paper's result is a reason to measure effort on your own evals, not a drop-in policy.
Third, the 42× figure is a maximum, not a typical bill. It is the largest mean-cost ratio on always-solved tasks across the models and benchmarks; the typical multipliers the paper reports in detail are 2.5×–5.3×. Quote 42× as the ceiling, not the expectation — and note the SWE-Bench Pro leakage caveat: the main cost figures use the original environment, where high effort's apparent gains partly reflect reading the gold fix.
Fourth, laddering pays for a full second run on every escalation. For workloads where nearly every task is genuinely hard, or where latency deadlines are strict, starting low and retrying can cost more than starting high. The policy wins because most real workloads are mostly easy tasks; if yours is not, measure before you switch.
Takeaways
- Reasoning effort is a credence good. You pay for hidden tokens whose necessity you cannot observe — even a solved task does not tell you the effort was needed.
- Max effort is not a superset. On Terminal-Bench, all 8 models lose lower-effort solves at higher effort; sol drops 76 → 73 going high → xhigh. Aggregate accuracy hides the forfeits.
- Easy tasks pay the hard-task price. On always-solved tasks, low → highest effort multiplies mean cost up to 42× (typically 2.5×–5.3× in the reported detail), via fatter calls and longer trajectories that re-bill growing history.
- Turning reasoning off backfires too. Luna at "none" takes 21.5 calls vs 5.4 at low and costs 4.58× more. The model compensates for missing deliberation by wandering.
- Laddering beats max on both axes. Start low, escalate only on verified failure: higher success on all models and benchmarks, median savings 25.3% (AIME), 41.5% (GPQA), 15.5% (SWE-Bench Pro corrected) — and more once waiting time counts.
- Check it yourself. The companion notebook recomputes every headline number — the cost multipliers, the 76 → 73 regression arithmetic, the none-effort trap, the laddering gains — plus a toy simulator showing why starting low wins when most tasks are easy.