Three sources of the thought state

A language model writes one token at a time. The token it just wrote is added to the text, and the model runs one more pass through its whole stack, for the position that token now occupies, to produce the one after that. Every token in an answer arrives from one of two places: the text in front of the model, or the numbers stored inside it. Which place supplied a given token is the question this module gives you the means to answer, and we start with two questions that force the two places apart.

The first question is one you could ask any model cold: “What is the capital of France?” Seven tokens in, and the answer, Paris, comes back at once. Nothing in the prompt says Paris. The fact arrives out of the model’s own stored numbers, because the pairing of France with Paris was in the training text millions of times, and we will watch the machinery hand it over.

The second question is built so that the same route is closed. Imagine you run a company called Aperture Instruments, which sells a digital camera called the C-40. A customer writes in asking how long they have to send one back, and you have pasted your returns policy above their question so the model can see it:

Returns policy, rev. 2026-02: Aperture Instruments accepts returns
of the C-40 within 45 days of delivery. Units returned after day 30
incur a 15% restocking fee. Opened lens kits are non-returnable.

How many days do I have to return the C-40?

The company is invented, and that is exactly what makes it useful. No model has ever read a word about the C-40, so nothing can arrive out of the model’s stored numbers without us noticing it arrive. Paris comes from those numbers; 45 has to come from the page. Two questions, one machine, and by the end of this part you will be able to say, for any token in an answer, which of the two supplied it.

The answer we want to the second question is You have 45 days to return it. Three tokens of that sentence are worth following, because each one gets into the answer by a different route:

• ␣days settles where it stands. English puts a plural after a number, so the token ␣days wins over ␣day; the number is right there, and no other part of the text has anything to say about it.
• 45 exists nowhere near where it is needed. It sits in the policy line, a few dozen positions back, and something has to carry it forward.
• whatever appears in place of 45 when you delete the policy comes from neither of those routes. Nothing in the text holds it, so the model supplies it out of its own expectations about what a returns policy says, the same route that supplies Paris, now filling a slot it has no fact for.

Same machinery all three times, and three different places the token came from. To see how one machine can do all three, picture it first at a size you can hold in your head. Each position in the text carries a list of three numbers. A stack of three blocks reads that list in turn, and each block adds its own numbers to it. At the last position, a vocabulary of five words is scored against the finished list, and one word is drawn from those scores.

A real model is that same object at scale: the lists are thousands of numbers long, the stack is dozens of blocks high, and the vocabulary runs to tens of thousands of entries. Every trip through it has the same four beats. The model reads the whole text so far, updates the numbers through the stack, compares the finished list at the last position against every word it could say, and draws one. Read, update, compare, draw. On the first trip the update runs at every position of the prompt at once; on every later trip it runs at the newest position only, and that position reads the work already stored for the others. The drawn token is added to the text, and the trip runs again for the token after it.

The finished list at the last position is the thought state, written h, and it is the only thing the vocabulary is ever compared against, so everything the model has read, remembered, or worked out has to be in it or it plays no part in what gets said. Three sources write into it on the way up, and by the end of this module they will have names: the baseline, the context term, and the memory term. The ␣days, the 45, and the 30 that replaces it are one token from each.

One statement about the numbers and figures in this module, made once. The structure of the pass and everything about the Aperture Instruments specimen are exact, and the token splits are those of OpenAI’s cl100k_base tokenizer. The entries shown inside example vectors, the probabilities in the softmax table, and the heights in the temperature bar charts are shaped to show the mechanism and were measured from no run. The figures draw fewer positions and fewer numbers than the real thing so that they fit on a page. The figure of three passes draws one word per pass, where the tokenizer cuts the answer into nine tokens with a lone space before 45. Everything else is stated as it is.

Watch External visual explanation

If you would rather see the whole pass in motion before taking it apart, Grant Sanderson’s Large Language Models explained briefly (8 min, 3Blue1Brown) covers the same loop at the same level of detail: text into numbers, attention, the feed-forward blocks, one draw. His name for what we call the thought state is the last vector in the sequence. A text version exists too.

tokenization turns text into vectors

The model receives text: your policy, the customer’s question, all of it as one run of characters with nothing marking where one ends and the other begins. Before any arithmetic can happen that text has to become numbers, which takes two steps.

First the text is cut into chunks. A fixed lookup table holds every chunk the model is allowed to recognize: between fifty thousand and two hundred thousand entries, roughly the headword count of a large dictionary, assembled once from the training text and never touched since. Your text gets spelled out of those entries and only those. Words that appeared constantly earned an entry of their own. Anything rare has to be built out of smaller scraps. The table is the tokenizer, and the chunks it produces are tokens.

The customer’s question comes apart like this:

["How"]["␣many"]["␣days"]["␣do"]["␣I"]["␣have"]
["␣to"]["␣return"]["␣the"]["␣C"]["-"]["40"]["?"]

Thirteen tokens. Read the end of that line closely. “C-40” is not one thing to the model but three separate tokens, C and - and 40, because Aperture Instruments and its camera did not exist when the table was built and nothing in it spells the product directly. Whatever the model says about the C-40, it assembles on the spot out of three fragments that mean nothing on their own.

Second, each token is looked up in another stored table and comes back as a list of numbers — 4,096 of them in a common mid-sized model, about eight thousand in a large one; we say eight thousand throughout this part. Thirteen tokens in, thirteen lists out. Each list also carries a mark of where its token sat in the order, so that 40 arriving tenth is distinguishable from the same 40 arriving second.

A list of numbers used this way is a vector, and the France question shows what one looks like from the outside:

Position 1 ("What"):    [  0.20, -0.50,  1.10, ... ]   (about 8,000 numbers)
Position 6 ("France"):  [  1.80,  0.40, -0.90, ... ]   (about 8,000 numbers)
Position 7 ("?"):       [  0.10,  0.30, -0.20, ... ]   (about 8,000 numbers)

Each slot tracks some abstract feature the model discovered during training. A positive value marks the presence of a trait, a negative value marks its absence or its opposite, and no slot has a name you could read off. Every position starts with a vector of this kind, and two tokens playing similar roles in text come out with vectors pointing similar ways. “Pointing similar ways” is arithmetic rather than metaphor, and that arithmetic is the operation the rest of the machine is built from.

Each slot in the token order is a position, and a position keeps its own vector for the whole trip. Thirteen positions, side by side, each carrying eight thousand numbers. That grid is everything the model has of your text: a block of numbers built a moment ago and thrown away when the answer ends. The characters, the words, and the policy as a document are gone; only the numbers made from them remain.

vectors climb a stack of identical blocks

Now the numbers travel. Each position’s vector is handed upward through a stack of identically shaped machines, forty of them in a small model and up to about a hundred and twenty in a large one. Each machine is a block, and they are where all the work happens.

A block reads the vector arriving at a position and adds to it. It computes a vector of its own, the same length, and adds that in slot by slot, so what leaves the block is the sum of what arrived and what the block contributed. The next block up receives that sum, reads it, and adds its own contribution to it. Then the next. All the way to the top. The vector under construction at a position is the residual stream, and it is built purely by addition: every block merges something in, and nothing is ever taken off.

Two consequences follow from adding rather than replacing, and both matter later.

Everything written stays legible. A block working near the top of the stack can still read what block 2 put there, because block 2’s contribution is part of the total that arrived. When a low block works out that 40 is part of a product name rather than a quantity, that finding is available seventy blocks later, while the answer is being composed.

And a later block can argue with an earlier one. Contributions can be negative, so a block that disagrees with what was written below it adds a contribution pointing the other way and cancels part of it. Nothing is erased, but a note can be crossed out by a later note.

Picture the residual stream as a noticeboard per position, with every block posting to it and nobody taking anything down: early blocks post what a token is, middle blocks post what it means here, and blocks near the top post something close to a prediction.

This is where our easy token settles. At the position holding 45, the next token needs to be a plural, the number that calls for it is right there in the same phrase, and no other part of the text bears on it at all. The blocks that push ␣days ahead of ␣day are low in the stack, the finding never gets contradicted higher up, and by the time the next token is being chosen it has been sitting on the noticeboard, unchallenged, for a long climb. Most tokens in most sentences are like this. They are the reason a model’s prose is fluent even when its facts are wrong.

context and memory converge on the same slot

The other two tokens cannot settle in place. Each of them reaches the answer by a different route, and you are about to watch both.

Take 45 first. It is in your text, but it is posted on the wrong board: it sits at the position holding 45 in the policy line, a few dozen positions behind the place where the answer is being written. Positions keep their own vectors, so nothing carries it forward on its own. Something has to go and get it.

That is what the first machine in every block is for. At every position it forms a short vector describing what that position is looking for, and another short vector describing what that position has to offer. Compare the first against the second at every other position and you get a score per position, saying how strongly these two line up. Where the score is high, the content of the matching position gets copied forward and added to the thought state here.

So the position composing the answer forms something that amounts to a number of days, attached to this product, that matches strongly against the policy position holding 45, and 45 gets pulled across into the answer’s noticeboard. The machine that does this is attention, and this is the only way information ever moves between positions. Every block runs dozens of these side by side, each looking for something different. Everything attention adds to the residual stream at the last position, summed over every block, is the context term, written h_ctx: the part of h that was pulled in from earlier positions.

Now delete the policy and ask again. The position composing the answer forms the same request, compares it against every position in the sequence, and finds nothing that matches: the customer’s question mentions the C-40 but says nothing about how long anything lasts. No number gets copied, because there is no number to copy.

What does the pass do with nothing to copy? It keeps running, because every stage is arithmetic that produces a value, and what comes out at the end is still a distribution over next tokens. A model can put its weight on a continuation that declines or hedges, as module 1.01 showed. When it does not, this is the route a fabricated number takes: the slot gets filled by the second machine in the block, which works at one position at a time: it compares the residual stream against tens of thousands of stored patterns and adds a blend of whatever those patterns are paired with. Those patterns were fixed during training, and among them is a great deal of general knowledge about how returns policies read. So a number arrives: 30, usually, because thirty days is what most of the internet’s return policies say. It arrives in exactly the same way, at exactly the same fluency, as 45 did when the document was there. This machine is the MLP, and roughly two-thirds of a model’s numbers live in it. Everything the MLP adds to the residual stream at the last position, summed over every block, is the memory term, written h_prior: the part of h that was injected from the weights, whether or not the context asked for it.

look first: the two lanes get the same request. Where does the second lane’s 30 come from?

Run the France question through the same two machines and you can see the memory term doing honest work rather than filling a gap. Attention at the last position gathers France together with capital; that is h_ctx, and it names the question without answering it. The MLP then recognizes that pattern in the residual stream and adds what was stored with it, a contribution pointing toward Paris; that is h_prior, and it is the answer. Same machine that supplied 30. The difference is that this time the stored pattern was the fact that was asked for.

And ␣days? It settled in place, which in these terms means that at the position holding 45, the position’s own starting vector, plus what the words just before it contributed, already leaned toward ␣days over ␣day before anything far away or anything from memory had a say. The starting vector is the baseline, written h_initial: what the token itself brought to the position when it came out of the lookup table.

Three tokens, three sources, and now one equation:

  h_initial   the baseline — the current token's own embedding
+ h_ctx       the context term — pulled in from earlier positions
+ h_prior     the memory term — injected by the MLP from the weights
─────────────────────────────────────────────────────────────────────────────
= h           the thought state, at the last position

That is bookkeeping rather than theory: the lookup table, the attention machines, and the MLPs are the only three things that ever add to a residual stream, so group the contributions by who made them and the sum falls out. For now, hold the picture: ␣days is mostly baseline, 45 is context, 30 and Paris are memory.

Sit with what the dashed path means. Both numbers reach the same slot by machinery running at the same fluency, and the finished sentence reads the same either way. Nothing in the finished sentence certifies which route supplied the number. Whatever signals a model holds about how familiar a subject is, they are not a calibrated check against a policy it was never shown, so the check that tells 45 from 30 sits outside the model: your policy, a test, or a person who reads both.