How reasoning models write their working down

a reasoning model re-reads the tokens it wrote

Two things changed to produce reasoning models. One is a training change, and the other is mechanical.

The training change: run the model on a problem, let it produce a long stretch of tokens before answering, then score the final answer with a program rather than a person. Mathematics, code, anything a checker can verify. Reward the runs that ended correctly. That method is outcome supervision, feedback on the final result alone, and it is the one this section describes. It is one method among others: process supervision gives feedback on each intermediate step instead.

Read what outcome supervision rewards, carefully. We will need it in a few pages, when we ask how much of a visible chain of thought you can actually believe. The score lands on the final answer and nowhere else. The tokens in the middle receive no direct supervision at all. One thing alone shapes them: whether the runs containing them ended correctly. Nothing scores their readability, and nothing checks them against what actually happened inside the pass that produced them. A trace that reaches the right answer by a route it does not mention scores exactly as well as one that narrates itself faithfully.

The mechanical change is the one that matters: those tokens go into the context. The model writes to its own prompt.

This is the way out the amphora leaves open inside the model. One pass is depth-limited, but a value written at token forty is readable at token forty-one, and token forty-one gets a fresh full-depth pass to build on it. Step-by-step work stops being capped by the height of the stack and starts being limited by how many tokens you will pay for and how many the context window holds.

The steps in that drawing are one token each. A real trace spends dozens of tokens per step, and the columns on the right are the same eighty blocks re-run rather than new ones.

answering in one pass versus many

Put the two ways of answering side by side, because the difference is easy to state now that you have the three sources in hand.

A standard model asked for the answer straight away answers with a single associative jump. It reads the prompt, builds one thought state at the last position, dots it against the vocabulary, and draws. When the problem needs several deductions in a row, that one jump has to carry all of them, and what usually fills the slot is the loudest association the MLP holds for the pattern in front of it — the memory term, arriving at full strength whether or not it is the right one. A thinking model routes through intermediate states before it answers. It writes each step down as tokens, and each later pass reads the earlier steps as context, so the context term is built up rung by rung instead of being asked to hold the whole ladder at once. A standard model asked to write out its steps climbs the same ladder. The thinking model has been trained to climb it without being asked.

Standard model — one hop:
[Problem] ─────────────────────────────────────────► [Answer]
              relies on the strongest immediate association

Thinking model — the step ladder:
[Problem] ──► [Step 1] ──► [Step 2] ──► [Step 3] ──► [Answer]
              builds context token by token; every step is a full pass

Here is a riddle old enough that you may know it, which is exactly why it makes the mechanism visible:

A man walks into a bar and asks the bartender for a glass of water. The bartender pulls out a gun and points it at him. The man says “Thank you” and walks out. What happened?

Watch the one-hop answer form. The context term carries bar, gun, water, and thank you, and the memory term is dominated by what co-occurs with bar and gun in ordinary text: hostility, a hold-up, a crime. The sum goes to the loudest association and the model leaps: “It was an attempted robbery.” The answer is wrong because the single jump skipped the constraint that ought to have governed it: nobody says thank you to a robber, and there was no second pass in which the constraint could be re-read against a candidate.

Now the step ladder. The model writes intermediate reasoning into its own context before answering. This trace is reconstructed, following the shape such traces take rather than quoting a run:

<think>
Step 1: Parse the core puzzle: why would someone say "thank you"
        after a gun is drawn on them?
Step 2: The gun caused sudden fear or surprise.
Step 3: What condition is treated by either a glass of water
        or a sudden shock?
Step 4: Hiccups. A glass of water cures hiccups; a sudden scare
        also cures hiccups.
Step 5: The scare solved the problem, so the water was no longer
        needed.
</think>
The bartender realized the man had the hiccups and cured them
by startling him.

Read step 1. The first thing the ladder does is write the constraint down — the thank you that the one-hop answer skipped — so that every later pass reads it as context and cannot lose it in the squeeze at the spout. Steps 2 and 3 narrow the search from bar and gun to water and shock, and step 4 is the memory term firing on a pattern it had not been shown until the trace built it: cured by water or by a fright. That pattern was never in the prompt. The model wrote it, then read it.

Grade this before you carry it. The riddle is famous, so a large model may hold its answer as a stored fact and produce hiccups in one hop; take the example as the shape of a class rather than a benchmark. The class is: problems where the loudest immediate association is wrong, and the right answer needs a constraint held across several steps. Date-and-threshold arithmetic on a returns policy is squarely in it, and that is what we watch next.

Nobody specified the behaviors that came out of this training. They were selected because they raise the chance of a correct final answer. The model spends more tokens on harder problems and fewer on easy ones. The architecture always allowed that, since a model ends its own answer by drawing the end-of-sequence token; what the training changed is that the model now tends to do it. It writes intermediate results down instead of holding them. It restates a result and checks it against a constraint. And it backtracks, which deserves a closer look, because a model cannot un-write a token. What it can do is write a correction, after which the corrected value sits in the prompt more strongly and more recently than the error. The wrong figure stays in the sequence; the correction outweighs it. That is the same weighted sum we watched in run four, where the losing number never left the context and was only outweighed. Here the model is weighing against its own arithmetic.

The specimen shows the same behavior. In a reconstructed trace of the C-40 refund, the model first counts from the order date, reaches 50 days, and calls the return outside the window. It then catches that the policy counts from delivery, recounts to 44 days, checks 44 against the 45-day window and against day 30 separately, and answers $4,165. This trace is reconstructed rather than captured: the beats follow the shape real traces take on date-and-threshold arithmetic, including the error and the catch, but no model produced these exact words.

Three mechanisms are visible in that trace, and each one is something we built earlier.

The backtrack worked because the wrong figure had been written as a token. Attention could read it, and a later pass could write a better-supported number beside it. Had “50 days” stayed inside the residual stream at the position that computed it, later positions could still have read it, but only through attention in the blocks at and above the height where it was worked out. Written as a token, it entered every later position at the bottom of the stack, where all eighty blocks could check it against the policy.

The re-read worked because the policy never left the prompt. Catching the delivery-versus-order distinction meant going back to the document after already computing something, and in one pass the constraint and the computation compete for the same eighty blocks. Across many passes the model can read, compute, and read again.

And in the trace the restocking fee survives with both comparisons against day forty-four written out separately, so neither comparison had to share the opening with the other six results. That is the explanation of run six this module has followed. The trace is reconstructed, so it shows the shape of that explanation and measures nothing. The measurement is one you can run: ask a model for the same refund twice, once answering directly and once writing the steps out, and compare whether the fee appears.

the context update at the end of the trace

There is one moment in a reasoning run worth slowing down for, because it is where the ladder pays out, and it is the moment budget limits break.

When the model writes the token that closes its scratchpad — </think> in the trace above — every step it wrote is still sitting in the context window. Nothing has been summarized or handed forward in a side channel. The pass that writes the first token of the visible answer runs attention across the whole sequence, the original prompt and the entire trace and the closing tag, and assembles a fresh context vector from all of it:

h_ctx = Attention( original prompt + thinking trace + "</think>" )

Now the query at the answer position is looking for the conclusion, and the keys it scores highest against are the positions where the trace wrote it. On the riddle, that is hiccups at one position, scare at another, cured at a third. Their values are blended into h_ctx, and the thought state at the answer position lands hard against the candidate vector for the word the trace arrived at:

ℓ("hiccups") = h · u_hiccups   →   p("hiccups") > 99%

That is the mechanism, and it has a failure mode you will meet in production the first time you cap reasoning tokens.

An unfinished scratchpad still gets read. When a reasoning budget runs out before the chain reaches a resolution, whether the framework closes the tag for you or the provider stops generation where it stands, attention at the answer position reads an incomplete scratchpad. The context vector it assembles points only toward the partial calculation, and the unembedding step has to make a low-confidence guess from unfinished premises. A truncated trace does not degrade gracefully into a shorter argument; it hands the answer back to the one-hop machinery, now with a half-built context in front of it.

So when you set a reasoning budget, set it for the shape of the task rather than for the average, and treat a trace that ends without a stated conclusion as a signal to re-run with room, not as an answer to ship.

a visible trace steers the answer without explaining it

A visible chain of thought looks like an explanation, and there is a measurement that says how far you can trust it as one.

The experiment is clean. Give a model a problem plus a hint at an answer, sometimes correct and sometimes deliberately wrong. Then check two things separately: whether the model’s answer moves toward the hint, and whether the visible reasoning mentions the hint at all.

That stops being a contradiction the moment you recall what outcome supervision rewarded. The tokens were shaped to reach correct answers. Nothing in that reward asked them to describe the path, so expecting fidelity is expecting a property the training never selected for.

What the trace closes and what it leaves open. A visible chain lets you catch arithmetic slips, wrong constants, and misread constraints, because those errors appear in the text and you can check them against the source. It does not tell you what else influenced the answer, and a fluent trace raises a reader’s trust without raising the answer’s reliability. Show it as working, never as warrant.

The habit: when you need to show a user why, generate something a machine or a person can check against the source. Quote the clause. Link the row. Print the calculation and re-run it. Surfacing the chain and labeling it an explanation puts weight on a beam that was never built to carry it.

where reasoning tokens are worth paying for

They pay off where the work is genuinely step-by-step and the result is checkable: mathematics, code, planning, constraint problems, anything with intermediate values worth holding. Run six is the shape.

They pay almost nothing on a single lookup. Run two does not improve with eight hundred tokens of deliberation, because there was never a depth problem there, only a routing problem. They pay nothing on tone and format.

And sometimes they cost you. On easy questions, a long chain can talk a model out of an answer it already had right, because more tokens means more chances to introduce an error and then build on it. On judgment and perceptual tasks, forcing an explanation can lower accuracy the same way describing a face makes people worse at recognizing it afterward: the reasons a person can articulate are not always the reasons that were working. And an early mistake is now sitting in the prompt as a strong, recent, readable signal for everything downstream. Backtracking catches some of those. Some of them get a confident structure built on top instead.

So sort the request before you spend on it. Sending all traffic through extended reasoning pays a large fixed cost in latency and tokens for a benefit most requests never collect, and a few are made worse by. That sorting step is a router: an entry-level agent that reads each incoming request, classifies it, and hands it to the specialist that should answer it, answering nothing itself. The same shape works here, with the classes being how much thinking a request deserves rather than which specialist it belongs to.

what you can do with this today

Route by the shape of the task. Lookups do not need reasoning tokens. Step-by-step problems do. Classifying the request first is usually the largest cost saving available in a production system, and it tends to improve quality at the same time.

Show checkable justification, not the chain. Quote the clause, link the row, print and re-run the calculation. If you do surface a trace, label it as work in progress rather than as an explanation.

When a model drops a step, give it somewhere to put the step. Before you reach for a bigger model or a firmer instruction, ask whether the answer required more results held at once than one opening can carry. If it did, the fix is to let it write them down.