How the memory term competes with context

the MLP injects the memory term

Take the France question to the middle of the stack. By then attention has assembled the context term, h_ctx, at the last position, and it carries France together with capital. The word Paris appears nowhere in the text. Attention can only move what is already in the sequence, so nothing it does can fetch an absent word. If the answer is to arrive, something has to supply it out of the weights.

What would a machine that does that have to look like? It has to recognize the pattern in front of it and hand back whatever was stored with that pattern, and it can use only what this position holds, because looking elsewhere is attention’s job. Here is the machine. The residual stream at this position is a vector of eight thousand numbers. Sitting in the block is a grid of about thirty-two thousand stored patterns, each a vector of the same length, fixed since training ended. Run the comparison down the grid, eight thousand numbers against eight thousand numbers, thirty-two thousand times over, and you get thirty-two thousand scores, each saying how strongly this position’s current state resembles that stored pattern. That one sweep is about a quarter of a billion multiplications, for one position in one block.

Then comes a step that sounds crude and is doing real work: every score that came out negative is set to zero. Recall what a negative score means on the compass. It means the residual stream points the opposite way from that stored pattern, and a row the state points away from has nothing to add, so it stays quiet. Only rows that genuinely matched survive the cut, and in practice a small fraction of the thirty-two thousand do.

Limits Where this model fails

The cut at zero is the ReLU, and it is the version that makes the mechanism easiest to see. Many current models use a smoother gate, GELU or a gated pair called SwiGLU, which lets a score just below zero through faintly rather than silencing it. A small fraction of rows still does nearly all the writing, so the picture holds; the exact count of quiet rows does not.

Now the write. Paired with every stored pattern is a second stored vector, its contribution: what this row adds to the stream when it matches. Take each surviving score, multiply its paired contribution by it, and add all of those into the vector at this position. A row that matched strongly writes a lot, a row that matched weakly writes a little, and a row that was zeroed out writes nothing at all. That sum is the memory term, h_prior. On the France question, the rows whose stored pattern looks like France together with capital fire, and what is filed under them points toward Paris. A grid of numbers fixed in training behaves as an associative memory, handing back whatever was filed under the pattern it is shown, and it does so for any pattern the stream can form, including ones assembled from text that did not exist when the grid was built.

one MLP, at the position writing the answer

arrives

the stream, carrying h_ctx: France + capital

assembled by attention in the blocks below

matches

the rows that look like this combination fire

negative scores zeroed · a small fraction of rows survive

writes

h_prior, pointing toward Paris

added into the stream beside h_initial and h_ctx

This machine is the multi-layer perceptron, almost always shortened to MLP and sometimes called the feed-forward layer. Each stored pattern with its paired contribution is a neuron: in the classic drawing of a network, the neuron is the single number the match produces, its activation, and the pattern is the set of weights running into it while the contribution is the set running out. “The neuron fires” means that number came out high. The MLP never looks at any other position; it works on the residual stream at one position alone. Roughly two-thirds of all the model’s numbers sit in these paired vectors, and this is where most remembered facts appear to live. When the prompt holds no copy of a fact, the pass depends on what training filed in these paired vectors to supply it.

Watch External visual explanation

How might LLMs store facts (23 min), chapter 7 of Grant Sanderson’s Deep Learning series, draws this machine for one fact, Michael Jordan and basketball: the rows as questions put to the vector, the cut at zero, the columns as what gets written back. It ends on superposition, which this module takes up under fact collision. The text version has the same walk-through.

one block, at one token position

in

the stream arriving from below

h_initial plus everything the lower blocks wrote here

attention

move information between positions → h_ctx

queries seek · keys answer · values get blended · dozens of heads at once

mlp

transform information at this position → h_prior

match ~32,000 stored patterns · zero the misses · add the mixture back

out

the same stream, two contributions larger

handed up to the next block, which does this again

Two machines, two jobs: attention moves information between positions and writes the context term, and the MLP reads one position’s residual stream and writes the memory term.

fact collision: a louder association outweighs the one asked for

Here is the case that shows why the three-way split is worth keeping in your head rather than just admiring.

Ask a model "What is the capital of Australia?". Attention does its work exactly as before: h_ctx at the last position carries Australia together with capital, and that combination is what the MLP is asked to recognize.

Now think about what the training text held. Every mention of Australia’s capital said Canberra. But text mentioning Australia at all, in travel, sport, business, and weather, said Sydney far more often than it said Canberra, and it said it near the same words. So two things sit filed in the frozen weights under a pattern shaped like Australia: a Canberra association attached to the capital part of the pattern, and a much heavier Sydney association attached to the Australia part. The MLP fires both. h_prior comes back as a blend, and in this toy the Sydney direction arrives at higher magnitude than the Canberra one.

Now the two terms are in tension. h_ctx says the capital, which favors Canberra. h_prior says the city that goes with Australia, which favors Sydney. If the memory term’s pull is stronger than the context term’s, the model writes the high-frequency distractor instead of the fact it was asked for. It has produced a part standing in for the whole, the famous city for the country. In the three-term picture, that failure is a fact collision: the memory term holds more than one association for the pattern in front of it, and the louder one outscores the one the question named.

Grade this honestly before you carry it. A large current model asked that exact question says Canberra, because the capital pairing is well rehearsed and post-training sharpened it. The collision shows itself on rarer facts, on smaller models, and on any question where the asked-for association is quieter than a neighbor that shares its pattern: the second-largest city, the previous CEO, the drug’s older name. The mechanism is the same everywhere. Nothing in the MLP checks that the loudest association is the one the question asked for, because nothing in the MLP knows what a question is. It matches patterns and hands back what was filed.

You have seen the softer version of the same situation already. Asked cold in the last module, with no policy in the prompt, the model asked where the C-40 was bought and what kind of product it is, and listed the usual windows: 30 days at Amazon, 15 at Best Buy, 30 or 90 at Walmart and Target. In the three-term picture, h_ctx carried the shape of the question and none of the answer, so every number in that reply came from h_prior: what the weights hold about return policies in general. That is fact collision with one side of the collision empty. Here both sides are full, and the wrong one is louder.

One caution before we go on. It is tempting to imagine a neuron for Canberra and a neuron for the C-40, each holding its concept. Models represent far more concepts than they have numbers to spare, so they pack them into overlapping directions instead of separate ones, and a single neuron will fire for DNA sequences, legal citations, and one flavor of Python syntax. That packing is called superposition. It is why you cannot find a model’s belief about something and edit it cleanly, and why techniques that try tend to damage neighboring facts.

one score is three votes summed

We have the three sources. Now watch them decide the word, because this is where the equation stops being a description and starts being a tool.

At the top of the stack, the finished vector h at the last position is scored against every token the model could say next. Each candidate token A has a stored vector of its own in the unembedding table, its candidate vector u_A, and its raw score, its logit, is the dot product of the two:

$$\ell(A) = h \cdot u_A$$

You know what that number means: how many traits the thought state shares with the identity of that word, agreements minus disagreements, weighted by how strongly both committed. Now use the fact that h is a sum. The dot product distributes over addition, so the score for any candidate is the sum of three separate scores:

$$\ell(A) = (h_{\text{initial}} \cdot u_A) + (h_{\text{ctx}} \cdot u_A) + (h_{\text{prior}} \cdot u_A)$$

Read that as three votes. The baseline casts one, the context casts one, the memory casts one, and the logit is the tally. Every candidate in the vocabulary gets its own tally, softmax turns the tallies into shares, and the draw is made.

That is exactly the picture we need for the Australia question. Here it is, with the shares taken over these four candidates alone:

Candidate	baseline vote	context vote	memory vote	logit	share
Sydney	+0.1	+1.4	+2.9	+4.4	56%
Canberra	+0.1	+2.6	+1.1	+3.8	31%
Melbourne	+0.1	+1.2	+1.6	+2.9	13%
Paris	0.0	−0.5	+0.2	−0.3	under 1%

Look at where each candidate’s strength comes from in this toy breakdown. Canberra wins the context vote, because capital is what the question said. Sydney wins the memory vote, because Australia is what the training text said most often. The baseline votes for nothing in particular, which is what a well-formed question’s baseline usually does. And the tally goes to Sydney, by 0.6 of a logit, which softmax stretches into a lead of nearly two to one.

Now the same picture for the France question, where the two big terms agree. The context vote for Paris is large, the memory vote for Paris is large, they add rather than fight, and the logit clears the runners-up by three points, +5.0 for Paris against +2.0 for Marseille. That is a lead of twenty to one after softmax. Within the three-vote picture, agreement between the context and the memory is how a confident right answer is built. Disagreement between them is how a fact collision is built. And a context vote that carries no fact at all, as in the C-40 question with its policy deleted, leaves the memory vote to decide alone: that is the picture’s account of a confabulation, and the pass runs to the end regardless.

Three votes, one score. Every logit is the sum of three dot products: what the token itself brought, what attention carried in from the context, and what the MLP injected from memory. In this picture, a right answer is usually the context and the memory agreeing, a confabulation is the memory voting alone, and a fact collision is the memory outvoting the context. The pass runs to the end in all three cases, and nothing inside it records which one happened.

what you can do with this today

The equation gives you a question you can ask in the middle of a working day, and it costs nothing. When an answer is wrong, ask which term was supposed to supply the right one: was the fact meant to come from the prompt, or from training?

If it was supposed to come from h_ctx, the question is whether the fact was in the text at all and whether anything pointed the machinery at it. Put the fact in, or move it closer, or remove whatever else in the prompt was competing for the same shares. This is what fixes the C-40 question.

If it was supposed to come from h_prior, you are relying on whatever training filed in the weights about it, with no signal telling you whether it is there or whether a louder neighbor shares its pattern. Rephrasing sometimes helps here, because a different phrasing fires different rows. Putting the fact into the context is the move that works, because it turns a memory term into a context term. It is not a guarantee: a well-worn association can still outweigh a fact sitting in a long window, and where the fact sits in that window matters too. This is what fixes the Australia question, and it is the reason the next module spends six runs on what the text around a question does.

And if the answer starts wrong from the very first token in a way that looks arbitrary, check the end of your prompt. h_initial is the smallest term, and it is the one you can shift with a single character.

Here is the whole pass on one card, in the vocabulary you now own:

1. INPUT
   Tokens: ["What", "is", "the", "capital", "of", "France", "?"]
   The current slot (?) starts from h_initial, its own embedding.

2. ATTENTION — communication between positions
   The query from "?" is dotted against every key.
   Shares: France 52%, capital 43%, others 5%.
   h_ctx is blended from the values and added to the stream.

3. MLP — memory retrieval at one position
   Stored patterns recognize "France" + "capital".
   h_prior, pointing toward "Paris", is computed and added.

4. RESIDUAL STATE
   h = h_initial + h_ctx + h_prior

5. UNEMBEDDING AND SOFTMAX
   Logit per candidate:  ℓ(A) = h · u_A  — three votes in one sum
   Probability:          p(A) = e^ℓ(A) / Σ e^ℓ(t)
   Small logit gaps become decisive shares.

6. ONE TOKEN OUT
   Drawn, appended, and the whole pass runs again for the next one.

In the next module we stop building and start predicting. Same weights, same question, six different arrangements of text around it, six different answers, and each one is a question of which of the three terms wins.