Six layers that hold a fact

Now we can answer the question every AI project argues about at least once: when the system needs to know something, where do you put it?

There are six places. They differ in what they change, how fast they take effect, what they cost, whether you can undo them, and whether the facts they carry can be trusted. Choosing the wrong one is the most expensive ordinary mistake in this field, and it is usually made in the first week.

a fact can live in one of six layers

layer 1: pretraining builds general capability

Building the model from random numbers. You feed it trillions of tokens of text, tens of thousands of years of continuous reading for a person, and repeatedly adjust every parameter so it gets better at predicting the next token. This is where the model’s language ability, its procedures, and its world knowledge come from. It is the origin of everything inside the stack: the facts stored in proportion to how often they appeared, the working algorithms training discovered because an algorithm predicts better than a lookup table, the narrow internal models of well-structured domains, and the prior underneath all of it.

You would do this to create a new foundation model, to add a language or a modality the base model never saw, or to continue pretraining on a genuinely underrepresented domain corpus: all of PubMed, all of a legal archive. Its cost grows with the number of training tokens and the model’s parameter count, which together set the compute, and then with the hourly price of the hardware, the runs that fail along the way, and the work of preparing the data.

Raising and educating a person from birth through university takes twenty years, costs a fortune, and produces a generally capable adult who knows nothing whatsoever about your company. That is this layer.

layer 2: fine-tuning installs behavior

Taking an already-trained model and continuing training on a small curated dataset of thousands to millions of examples, where pretraining had trillions. It covers supervised fine-tuning (imitate these input and output pairs), preference training such as RLHF or DPO (prefer this response over that one), and parameter-efficient methods like LoRA that train a small add-on instead of the whole model.

A team gathers its company documents and fine-tunes a model on them, expecting a model that knows its business. Then it asks the model about a policy written in those documents.

Fine-tuning installs behavior well: consistent output format, tone, domain vocabulary and reasoning patterns, reliable tool-call syntax. It is also how you compress a long system prompt into the weights to save tokens on every call, and how you distill a big model’s behavior into a small one.

It installs facts badly. Fine-tuning can encode a fact; the difficulty lies in how weakly it is stored and in what it takes to check or change it. A fact learned from a small corpus is stored weakly, and it has to compete with the model’s default sense of what a document like this usually says — the prior. In run one, the first of 3.07’s thought experiments, we asked the bare question with no policy in front of it, and the model filled the gap with the most ordinary return terms available: thirty days, original packaging, none of them ours. Those defaults were built from trillions of tokens, and a few thousand fine-tuning examples rarely outweigh them. A fact that does take hold has no source standing behind it, so the only way to check it is to ask the model and compare, and changing it means another round of training.

Nor can it be corrected surgically, and the reason is worth restating. A model needs to represent far more concepts than it has numbers to hold them in, so it packs several concepts into overlapping directions instead of giving each one its own — the packing we called superposition. A single neuron fires for DNA sequences, legal citations, and one flavor of Python syntax. There is no location on disk holding the model’s belief about your product, so there is nothing to open and edit, and every attempt to force a specific fact in bends whatever else was sharing the space. Editing methods that aim a change at one association exist in the research literature, and they stay too blunt about the neighbors to lean on in production. Deleting a fact outright is not on offer.

An apprenticeship is the right comparison. The person is already educated; now they learn your house style, your terminology, how your team writes a report. Weeks rather than decades, and installing one new fact still means another round of training rather than ten seconds of telling them.

layer 3: prompting steers the input, instantly

Text you put in the context window to steer behavior. The weights are frozen and identical for everyone; you are only changing the input. This covers system prompts, instructions, worked examples, and structured reasoning requests. It is the layer an agent definition takes apart block by block: the prompt written as named sections (who the agent is, what it may and may not do, the format its answers take, the examples that show it) so that a general model is committed to one specific job by text alone.

Prompting is where you start, always, because it is free and instant. It costs tokens on every single call, long prompts degrade instruction-following, and none of it survives to the next conversation.

The task ticket you hand someone at the start of a shift is the same thing. “Write this as a formal memo, keep it under a page, here’s an example of what good looks like.” It costs nothing, works immediately, and they forget it the moment the shift ends.

layer 4: grounding puts outside facts in the window

Inserting authoritative source text into the context so the model answers from documents rather than from its weights. The standard implementation runs in four steps. Cut your documents into chunks. Turn each chunk into a list of numbers so it can be compared against a question. Pull back the chunks scoring closest to the question being asked. Paste those into the prompt above it. That arrangement is called retrieval-augmented generation, and it is usually paired with a demand for citations.

In run two we put the policy document above the question, and the answer came back quoting our 45 days instead of inventing a plausible window. Grounding is that move built as a system: rather than you pasting the right document in by hand, a retrieval step finds it for you on every question. Use it for any question over private, proprietary, or recent data, and anywhere you need to show where an answer came from.

Its key property is the one fine-tuning cannot offer at any price: the facts live outside the model. You can add, correct, or delete one instantly, and no retraining is involved. Hand someone the case file before they answer, and they read your documents and cite page numbers rather than reciting from memory. A page number gives you a place to check the claim, and someone still has to open the file and compare the two. Update the file and their next answer changes.

layer 5: tools reach live data and change the world

Giving the model a menu of functions it can invoke. The model executes nothing itself; it emits a structured request (call get_weather with city=Austin), your code runs it, and the result comes back into the context for the model to use.

Reach for tools when you need live data, exact computation where a calculator beats a language model, search, or an action in the world: sending mail, writing to a database, running code.

The line between tools and grounding is worth holding. Grounding is read-only lookup of text you prepared in advance. Tools are arbitrary computation and can have side effects. Search-as-a-tool is where the two blur.

Give the worker a calculator, a phone, and keys to the warehouse and they can now reach things they do not personally contain. Unlike every layer below, they can also change things. That is why the trust boundary matters more here than the wiring does: connecting a tool is a few lines of configuration, while deciding what an agent may reach, what a returned result is allowed to do, and which actions need a human before they run is the part that keeps the reach safe to grant.

layer 6: memory is grounding on a store the system writes

State that persists across sessions. The system writes down facts, preferences, and history, then retrieves the relevant ones into the context on future turns. Mechanically it is usually grounding pointed at a store the system itself writes to.

Use it to remember a user’s name, preferences, ongoing projects, and past decisions, and to give a long-running agent continuity without training a model per user. The design questions are what gets written, what gets forgotten, how conflicts resolve, and whether the user can inspect and delete it. The Patient Advocate carries a patient’s own medical history between appointments, where forgetting a diagnosis or a prescription between sessions defeats the entire point of it, and it shows where each answer lives. Two pieces settle the first three. A filtering step reads each finished conversation and keeps only the handful of facts that would still matter next month. A written doctrine in the Advocate’s prompt then governs the other side: how a recalled fact must be cited when the agent uses it, and which one wins when a newer fact contradicts an older one. Tooling around the store answers the fourth: an operator can list the records stored for one user and delete them one at a time, or erase that user’s store whole.

The notebook a worker keeps between shifts is the whole of it. Nothing about them changed overnight. They opened the notebook and read what happened last time.