What is a reasoning model? When it’s worth letting the AI think
Thinking took three models from 4 right answers out of 9 to 8, and turned a one-line rewrite into 496 tokens. When it’s worth it.
On this page
Ask a chatbot which months of 2026 start on a Sunday, and it may answer in a second, sound certain, and be wrong. Ask the same model again with “thinking” switched on, and it pauses for a few seconds, then gets it right. Nothing about the model changed between the two answers. The only difference is that, the second time, it was allowed to work the problem out before replying.
That pause is what a reasoning model is. Every major AI vendor now sells one, and most chat apps show a “thinking” label while it runs. Thinking makes a model more accurate on problems with several steps, and you pay for every word of it, even the ones you never see. This post explains what happens during that pause, shows a small test we ran on three models, and ends with a simple rule for when to switch it on.
A reasoning model writes out its working before it answers
A language model writes one small piece of text at a time, each piece chosen from everything written so far. Our token explainer covers those pieces in detail. The catch is that a model gets roughly one step of effort per piece. If the very first word of its reply has to be the answer, it has to produce that answer in one jump.
Thinking gives it somewhere to put the in-between steps. Before the reply, the model writes a scratchpad: it restates the problem, tries an approach, checks it, and sometimes backs up. Then it writes the answer you see, with all that working already on the page behind it. A schoolchild doing long division on paper beats the same child doing it in their head, for the same reason.
The idea started as a prompting trick. In January 2022, Jason Wei and colleagues at Google showed that adding worked examples to a prompt made models solve maths word problems far better. With eight step-by-step examples, PaLM 540B set a new state of the art on the GSM8K benchmark. They called it chain-of-thought prompting.
Reasoning models bake that habit in through training. The clearest public account is DeepSeek’s R1 paper from January 2025. The team rewarded a model only for reaching correct, checkable answers and let it work out how. Its score on the AIME 2024 maths exam rose from 15.6% to 71.0% over training, close to OpenAI’s o1, and long, self-checking working emerged on its own along the way.
In our test, thinking fixed four of five wrong answers
To see the effect without a benchmark’s polish, we asked three models three short questions, each with thinking off and then on. The questions sound easy and are not: which months of 2026 begin on a Sunday, what date falls 100 days after Wednesday 21 October 2026, and how many times the letter r appears in “The strawberry farmer’s daughter carried four ripe raspberries.” Every prompt asked for the answer and nothing else.
With thinking off, the models got 4 of 9 right. With it on, they got 8 of 9. Qwen3.8 27B, an open model, missed all three without thinking and got all three with it. It even had the right date for the 100-day question and the wrong weekday, a typical one-jump slip.
The cost showed up in the clock as well. With thinking off, the nine answers took between 0.8 and 2.6 seconds each. With it on, they took between 1.9 and 7.1 seconds, and each model wrote between 238 and 647 hidden tokens before a reply that was often a single word. For a question where the first answer was already right, those seconds bought nothing.
One result is worth a closer look. Claude Haiku 5.5, with thinking off, ignored our request for a bare answer and worked step by step in plain view anyway: 233 visible tokens to count the r’s. It still got 14, and with thinking on it got 12. Thinking raises the odds of a right answer; it does not guarantee one. The true count is 13, and a three-line program would never miss it.
You pay for every hidden word, even when you only see a summary
The working is not free, and most of it is hidden. OpenAI’s reasoning guide says reasoning tokens are not visible through the API, take up room in the context window, and are billed as output tokens. It suggests reserving at least 25,000 tokens for reasoning and output when you start out. If a request hits its limit mid-thought, you can pay for the thinking and get no answer at all.
Anthropic and Google say the same thing in their own words. Claude’s thinking documentation shows you a summary of the thinking but bills the full thinking tokens Claude generated. Google’s Gemini thinking guide prices a response as output tokens plus thinking tokens. Output tokens are the expensive kind, often several times the input price, so thinking lands on the costliest line of the bill.
In our test, one polite rewrite of a seven-word sentence cost Qwen3.8 27B 12 tokens with thinking off and 496 with it on, for an answer of the same quality. That is about 40 times the tokens, 39 times the money and three times the wait. The money is tiny here: the 27 calls in this whole test cost 3.3 cents. Multiply it by thousands of requests a day, or by an agent that loops for an hour, and it stops being tiny. Our cost-per-task post explains why the price per token alone hides this.
It pays off on maths and logic, and barely anywhere else
The best evidence on where thinking helps comes from a 2024 study by Zayne Sprague and colleagues, “To CoT or not to CoT?”, published at ICLR 2025. They pooled results from more than 100 papers and ran their own tests across 20 datasets and 14 models. The pattern was blunt.
Symbolic reasoning gained 14.2 points on average, maths 12.3 and logic 6.9. Everything else, from commonsense questions to reading comprehension, moved from 56.1 to 56.8. On the broad MMLU knowledge test, the authors traced as much as 95% of the benefit to questions that contained an equals sign in the question or the model’s reply. Thinking helps when there is something to calculate or check, and not much otherwise.
The same paper found a second limit. Much of the gain came from the model doing calculations more carefully in its working, and on those problems it still did worse than handing the calculation to a proper tool, such as a code interpreter or a solver. Thinking is a better way to do arithmetic in prose. It is not a better calculator.
That matches common sense. Thinking cannot add a fact the model never learned; it can only rearrange what is already there. If a model does not know a date or a name, more working just gives it more room to reason confidently toward a wrong answer. Looking the fact up beats thinking about it.
Easy questions get overthought, and you still pay
The opposite failure is common too. A December 2024 paper with the memorable title “Do NOT Think That Much for 2+3=?” measured how early reasoning models handled trivial questions. Asked for 2 plus 3, they used on average 1,953% more tokens than ordinary models. In one example, QwQ-32B-Preview had the right answer 39 tokens in, then kept going to 901 tokens and 13 separate solutions.
Vendors have since added ways to rein this in, but the habit has not gone away. In our test, Gemini 3.8 Flash spent 786 hidden tokens and 8.4 seconds on the one-line rewrite above. Its route on OpenRouter rejected our request to turn thinking off. Some models now think on every call, and the only control left is how much.
The time matters more than the money for most people. A short reply that takes eight seconds feels broken, and in a chat that adds up quickly. If you run models on your own machine, thinking costs nothing in fees and everything in seconds: a laptop that writes 20 tokens a second needs 25 seconds to get through 500 tokens of thinking before the first word of the answer appears. Our guide to local AI on every size of machine has real speeds for common hardware.
Every major tool now has a dial, so set it per task
The good news is that thinking is no longer a separate product you have to pick. Most current models are hybrids that can think a lot, a little, or not at all, and each vendor exposes that as a setting.
The settings differ in style. OpenAI exposes reasoning.effort, with levels running from none to max depending on the model. Anthropic’s current models use adaptive thinking: Claude decides per request whether to think, and an effort setting steers how much. A simple factual question may get no thinking at all. Google’s Gemini models take a thinking_level, and its docs advise lowering that level, rather than cutting the output limit, to save money without truncated answers.
On your own machine, the same control has been in Ollama since May 2025. Run a model with --think or --think=false, or type /set think and /set nothink in a chat session. Qwen3.8 27B, the open model from our test, is an 18 GB download with thinking support, released under the Apache 2.0 licence. In CSuite, the bundled local runtime runs the same chat actions as the cloud providers, so moving a task between them is a model switch.
Think on the hard steps, answer fast on the rest
The rule fits in one sentence: switch thinking on when a task has several steps that depend on each other and a wrong answer would cost you something. Budgets, schedules, code, dates and anything you would work out on paper qualify. Rewriting, translating, summarising, chat and simple lookups do not, and thinking there buys you a slower, pricier version of the same reply.
If you build on these models, the same rule becomes a default. Start every request at the lowest effort the model offers and raise it only for the task types that earn it, rather than paying for medium everywhere. Keep the output limit generous when thinking is on, since a cut-off can bill the thinking and return nothing. Then log the reasoning token counts the APIs report, so the cost is something you see in a dashboard rather than discover on an invoice.
Two habits make this work. First, change the default: defaults often sit in the middle (Claude Opus 5.5 starts at medium effort), and most of your requests need less. Second, when an answer matters, check it the way you would check a colleague’s, because thinking makes a model more careful, not infallible. In our small test the best case was 8 of 9, and the miss was a count a calculator would never get wrong. For anything you can compute, compute it; save the thinking for the steps a calculator cannot take.
