The Architecture of Prediction

Modern computation rests on a simple principle: any system can be described in ones and zeros. This is binary logic: what is (1) and what is not (0), true or false, present or absent. Reduce a problem to those discrete states and you get the exact, repeatable precision we have used to compute everything from a bank ledger to the turbulence over a plane’s wing.

However, our lives rarely offer clean binaries. We live in the gray, among things that are mostly true, shaped by perspective, and open to interpretation. Fortunately, long before we built binary machines, we built the tool for working among the gradations of opinion: language. Language is humanity’s method for capturing functional truth about a messy world. It draws a smooth circle over what is really a polygon with a million sides, inexact by design and close enough to act on, and it lets one mind hand that working approximation to another.

For decades, the two worlds met on the machine’s terms. Using a computer meant translating fluid human intent into strict programming syntax, where a single misplaced character halts the system. The computer carried the arithmetic; the human carried every bit of nuance, context, and intent.

A large language model bridges that divide. It is a system trained on massive amounts of text until language itself became computable: rather than bending human thought into machine code, the computer reads and produces language directly.

A traditional computational function is deterministic: same input, same output, every time. A language model computes probabilities. Give it an input, a question or a prompt, and it scores its entire vocabulary on what is most likely to come next, then produces its answer one token at a time, a token being a word or a fragment of one. And by design it does not always take the single most likely candidate; it draws from among the high-probability options, which is why the same question can come back worded three different, coherent ways.

That drawing of new text, token by token, is what the “generative” in generative AI names. The model composes sentences that appear nowhere in its training data, and the result is a conversational layer between human reasoning and the digital systems underneath.

A large language model does one thing: it predicts the next token. Everything an agent appears to be, a code reviewer, a clinical assistant, a research strategist, is built on top of that single calculation, run over and over. We come back to this rule in every module of the course.

To predict each token, the model relies entirely on its inherent weights, shaped by training across trillions of words of human writing. A dataset of that immense scale forces the network to absorb deep structural patterns in how the world is described. Yet that same training gives it no internal signal to distinguish between patterns it saw a billion times and those it barely encountered at all.

Think about those two facts side by side: the prediction is excellent, and nothing in the machine marks where prediction shades into invention. Together, they explain both the remarkable capabilities of modern models and their most catastrophic errors. Once you understand this mechanic, failures stop being surprises and become a short list you can design around: three ways the guess goes wrong, and two loops that catch it from outside.

what happens when you press send

Here is the entire process first, so that everything after it has somewhere to attach. Every bold term below gets its own explanation later; read this as a map, not as a list to memorize.

You type a question and press send. What reaches the model is plain text, and the first thing that happens is that a fixed lookup table cuts that text into the tokens from the opening: sometimes a whole word, often a fragment of one, sometimes a single mark of punctuation. The model never sees your letters, and it never sees your words as words. It sees a sequence of tokens.

Those tokens go into the network, a stack of stored numbers called weights, fixed when the model was trained and identical for every person using it. The network reads the whole sequence and produces one thing: a score for every token it could possibly say next, across its entire vocabulary. Not a sentence. Not an answer. One score per candidate token.

Those scores are turned into probabilities, and one token is drawn from them. That single token is the first piece of your answer.

Then the whole thing runs again. The chosen token is stuck onto the end of the text, and the model reads all of it, your question plus the one token it just produced, and guesses the next one. A two-hundred-word answer is roughly three hundred of these passes, each one a complete re-reading of everything so far. Call one trip round that cycle a pass; the word comes back in every module of this course.

Two consequences fall out of that loop, and both matter more than anything else in this module.

The first is that everything the model can take into account has to be in the text. There is no side channel, no memory, no notes. Whatever you want the model to use, you put in the sequence. Everything the model can hold at once is called the context window, and it has a hard size.

The second is that nothing is written down at the end. The weights that produced your answer are the same weights afterwards. The model did not learn your name, remember your correction, or update itself. Close the conversation and every trace of it is gone.

Why is that simplification safe to carry? Because the cache changes how fast the pass runs and nothing about what the pass can see. Whatever you take from this module about the window, the weights, and the three failures holds with or without it.

Now we build that map up properly, starting with the stored numbers.

the machine that makes the guess

weights, and where they come from

An artificial neural network is a large pile of arithmetic arranged in layers. Numbers go in at one end, get multiplied and added their way through, and numbers come out the other end. Each connection inside scales the signal passing through it by a stored value, and that stored value is a weight.

A modern model holds tens or hundreds of billions of weights. Together they are the entire content of the model: its language ability, its apparent knowledge, its habits of phrasing. There is nothing else in there.

Where do those numbers come from? Not from anyone choosing them. They come from training, which is mechanical and, at its core, simple. Show the system a piece of real text with the next chunk hidden. Let it guess. Compare the guess to what actually came next, and measure how wrong it was. Then nudge every one of those hundred billion numbers a tiny amount in whichever direction would have made the guess slightly better. Repeat across trillions of examples.

Nobody designs the structure that results. It is whatever reduced the error.

The material is not only prose. Frontier models train across source code, digitized books, transcripts, images, and structured tables. Even so, the network stores no lookup table and no database of facts. Follow what the nudging does to a fact. A fact that appeared thousands of times in the training text was nudged into the weights thousands of times, and it is strongly represented there. A fact that appeared twice was nudged twice, and it is weakly represented or not at all. And nothing in the procedure ever wrote down which case a given fact is in: the guess was scored, the numbers moved, and no record of “well rehearsed” against “barely seen” was kept anywhere. A well-rehearsed fact and a barely-seen one come out of the same arithmetic, at the same speed, and the model holds no internal certainty score that separates them. That is the last line of the derivation, and all three of this module’s failure classes come from it.

Two sets of numbers were in play just now, and the split between them is the second thing to hold. The weights are one set, and we have just seen that they were fixed when training ended. Everything else in the pass, your question, your documents, the answer so far, is the other set, built fresh from your text on every pass and, as the press-send loop showed, thrown away when the conversation ends.

Weights vs. context: the weights are stored, shared across all users, and fixed at inference time. Everything else, your prompt, your retrieved data, and the tokens generated so far, is ephemeral text held in active memory during the session and discarded when it ends. When evaluating what a model “knows,” always check whether that knowledge is assumed to be in the weights or provided directly in the active context.

Here is a two-minute experiment that makes the split concrete. Tell a model something it cannot possibly have been trained on, such as the name of your dog. It will use the name happily for the rest of the conversation. Now open a completely fresh conversation and ask what your dog is called. It has no idea. Nothing was learned; the name was sitting in the text, and the text is gone. Every memory feature you will ever meet is engineering built on top of that fact, and in module 2.05 you will build one yourself, as a filing system around the model rather than a change to it.

tokens: what the model actually reads

Before any of that arithmetic can run, your text has to be cut into pieces of a fixed, known kind. That cutting is done by a lookup table built once, before training, and never changed afterwards. It holds somewhere between fifty thousand and two hundred thousand entries, and each entry is a token.

Tokens are not words. The table is built by finding the chunks that appear most often across a huge sample of text, so common short words get an entry of their own, while longer or rarer words are assembled from several pieces. The word “unbelievable” typically comes apart into three or four tokens rather than arriving whole. Punctuation marks usually get their own token. A leading space is generally part of the token that follows it, which is why ” the” and “the” are different entries.

Why does the unit matter to you, if you never see it? For three practical reasons, and you will meet all three again.

You pay per token. Pricing, rate limits, and speed are all measured in tokens rather than words. As a rough conversion for English prose, a hundred tokens is about seventy-five words, so a thousand-word document costs you roughly thirteen hundred tokens. Keep that ratio in your head; it is the one you reach for whenever you size a prompt.

The context window is measured in tokens too, which we come to next.

And the model cannot see inside a token. It has no direct access to the letters, because letters are below the resolution of what it reads. The well-known demonstration is asking a model how many times the letter “r” appears in “strawberry” and watching it answer confidently and wrongly. Many current models now get this right, because post-training taught them to spell the word out letter by letter first, which converts the problem into one they can see. Grade that honestly: the underlying limitation has not changed at all; it has been routed around. Any task that depends on counting characters, reversing a string, or reasoning about spelling is working against the grain of how the model reads, and the practical move is to give the model the letters as tokens, spelled out, or to hand the counting to code.

the context window

Everything the model can consider on a given pass has to fit inside one budget, measured in tokens, called the context window. Your system instructions, every retrieved document, every previous message in the conversation, and the answer being generated all share that one budget.

Modern windows are large. A 128,000-token window holds roughly 96,000 words at the ratio above, which is a 350-page book. That sounds like enough room to stop worrying, and in agent systems it very often is not: a long-running agent accumulates tool outputs, retrieved documents, and its own previous turns, and those grow much faster than a conversation does.

What happens when you exceed this budget depends on the environment. At the raw API level, exceeding the limit throws an error. In chat interfaces and agent frameworks, the system manages the boundary dynamically by discarding older text, and that case is the one that catches people. Follow the mechanism through. The pass reads the sequence in the window and nothing else; there is no side channel, as we saw at the top. So a token that no longer fits is, to the model, a token that was never written, and nothing in the pass compares what it can see against what it once could, because the live text is all there is to compare against. The loss produces no error and no change of tone.

Context overflow: when conversation history and reference documents exceed the context window, systems configured with rolling history silently drop the oldest tokens out of scope. The model produces no warning; it simply continues generating answers without access to the dropped instructions.

Picture the failure concretely. You set up an agent with careful instructions at the top of the conversation. Forty turns later, those instructions have scrolled past the edge of the window. The agent keeps answering, fluently and confidently, with no access to the rules it was given, and nothing anywhere reports a problem. The behavior change looks like the model “forgetting” or “drifting,” which is why people reach for prompt rewrites when the actual fix is a window budget.

The habit that follows is arithmetic you can do before a session starts. Add up the instruction, the documents a turn will retrieve, and the size of a typical tool result, and divide the window by that sum; the answer is roughly how many turns you have before the oldest text starts falling out of scope. Where you cannot measure it, budget it, and put the instructions somewhere the trimming cannot reach.

three ways a language model fails

All three failures below come from one property established above: the model produces the statistically likely continuation, and it holds no internal certainty score for what it does and does not know. Once you see them as three faces of that one fact, you stop treating them as unrelated bugs, and you start asking one question of each wrong answer.

confabulation

A model asked for something it does not hold will not stop, and will not say so. It produces the text that best matches the shape of a correct answer.

The industry usually calls this hallucination. This course uses the older word from psychology, confabulation, because it names the thing precisely: a person confabulating is not lying. They sincerely report a memory that was never formed, assembled on the spot from what would plausibly have happened, and they are as confident as they are about real memories.

Ask a model for a court precedent on an obscure property dispute and you may get a citation complete with volume number, judge’s name, jurisdiction, and a quoted passage, for a case that never existed. Every surface feature of authority is correct because the model has read tens of thousands of real citations and knows exactly what one looks like. The only thing missing is the case.

Why is that dangerous rather than merely annoying? Because there is no signal to catch. A fabricated citation and a real one are produced by the same process, at the same speed, with the same fluency and the same confidence. Nothing in the output distinguishes them, which means every check has to come from outside the model. That is the absent certainty score showing on the surface of the text, and it is the point people misjudge most often.

Fluency is not accuracy: a model’s apparent confidence reflects how clearly a linguistic pattern matched its training data, not whether the claim is factually true. High fluency often masks complete invention.

the grounding gap

The second failure is about contact with the world. A model works entirely inside a closed system of symbols, learned from descriptions. It has read every account of water ever written, and it has never been wet.

So it will cheerfully tell a camper to boil water over an open flame in a thin plastic bag, without noting that the bag melts. It will write code calling a database function that was retired two years ago, because that function appears throughout the text it learned from and nothing told it the world moved on. Both errors are the same shape: a statement that is perfectly consistent with the descriptions the model absorbed, and inconsistent with present physical or system reality.

This one has a straightforward fix, which is the whole reason the rest of the course exists. Give the model contact with reality by putting reality in the text.

systemic bias

The third failure is inherited. The network mirrors the statistical skews of what it read, and historical prejudices and representation imbalances in the source documents carry straight into the weights.

Make it concrete. An automated hiring agent trained on ten years of a company’s own resume data learns what that company’s past decisions looked like. If those decisions systematically favored certain backgrounds, the agent reproduces the pattern, assigning lower fit scores to qualified candidates whose work histories carry markers associated with underrepresented groups. The model is doing what it always does, which is to continue the pattern it was shown. That is exactly the problem.

Bias differs from the first two failures in one important way: prompting and grounding alone cannot reliably eliminate it. You can instruct a model to ignore demographic indicators, and the underlying statistical skews remain in the weights. It has to be addressed at the system design level, measuring outcomes in aggregate, auditing across groups, and constraining what you allow the agent to decide at all. Where the first two failures show in a single answer, this one shows in aggregates, which is why the check for it is a measurement across many outputs rather than a read of one.

On your first day, watching for three separate failures is exactly the right habit, and it is what the agent work of Part 2 asks of you. The mechanism modules ahead in this part open the machine and show that, underneath, they are one mechanism seen from three angles: every answer is drawn from a sum of what the prompt supplied and what the weights supplied, and you will be able to say, for any wrong answer, which of the two was supposed to supply the right one. Nothing you learn here has to be unlearned there.

grounding: putting reality in the text

Grounding is the practice of supplying verified reference material, database records, source files, policy documents, or live system state, directly into the context window before the model generates a response.

Grounding works because the model’s predictions are conditioned on everything in the active sequence. Ask about a company’s return policy without providing the text and the model generates the plausible shape of a policy, synthesized from thousands of generic examples in its training data. Paste the actual policy into the prompt and that specific text dominates the context window, steering the next-token probabilities toward the exact terms, dates, and numbers in the document. In module 1.03 we trace this token by token, and you will watch the number cross from the pasted policy into the answer, and watch what fills the slot when the policy is missing.

Two engineering advantages follow. Facts live outside the model, so you update information by editing a document or querying a database rather than retraining a network. And grounded systems let you require citations, which converts an unverifiable claim into a statement tied to a source line you can inspect.

To see the failure mode grounding prevents, look at the public legal record. In 2023 a New York attorney used an ungrounded ChatGPT session to find supporting precedent for a federal brief. The model returned convincing citations with court names, volume numbers, and quoted rulings. Six of the cases did not exist. The exchange below is quoted from the attorney’s sworn affidavit in Mata v. Avianca. Grade the instrument as you read: the opening request is paraphrased from the research task, and the two turns that follow are verbatim. Notice what happens when the ungrounded model is asked to verify its own fabricated citation.

This is why factual checks must sit outside the model, and grounding is the most direct way to enforce them. Placing verified reference documents in the prompt anchors generated text to the specific source in front of the model rather than to generic statistical patterns in its weights. Requiring exact citations gives you an audit trail you can verify immediately.

Understand what the technique actually does. It does not alter the underlying weights or make the model inherently truthful. It structures the context window so that extracting facts from the source, or stating plainly that the source lacks the answer, becomes the highest-probability continuation. Accuracy is an engineering constraint you build from the outside, never an internal property of the model. Writing prompt rules that enforce this behavior is the craft of module 2.01. The final safeguard remains yours: before you rely on generated output, open the citation and check the primary source.

who checks the work

Because these errors come from the mechanics of the machine rather than from a mistake anyone made, checking output is part of the architecture rather than a courtesy. Someone or something outside the model has to look, because a model with no internal certainty score cannot do it from inside. Two loops do the looking, one built around your judgment and one built around volume.

the human in the loop

For individual decisions and low-volume systems, that someone is you, working at two distinct stages.

At the input stage, you control what enters the window: stripping credentials and sensitive data that should never be sent, and deliberately placing the grounding material the answer will need. Most output problems are input problems, and this is the cheapest place to solve them.

At the verification stage, you check what came out: auditing claims against sources, following citations to confirm they exist and say what the model said they say, and testing logic rather than accepting fluency as evidence of it.

The discipline sounds obvious and is genuinely hard to sustain, for a reason worth naming so you do not mistake it for carelessness. Fluent, confident text is persuasive, and reviewing it is boring. After forty correct outputs, the forty-first gets less scrutiny, precisely when a wrong one is most costly. Building the check into the system, a step that must be completed, a field that must be filled, beats relying on attention that will naturally fade.

the agent in the loop

When a system runs thousands of operations a minute, no human reviews every turn. People can only inspect samples. So you place a second model in the path and give it one job: inspection.

A review agent sits between the primary generator and the user. It takes the draft, evaluates it against strict rules, a required output shape, or a safety policy, and then either passes it, halts it, or sends it back to the generator with specific correction instructions.

Why a second model, rather than asking the first one to check itself? Because of that absent certainty score again. Asking the generator to check its own work in the same pass gets you a fluent explanation of why the draft is fine, because that is the statistically likely continuation of “review your work.” The property that makes the review agent work is separation: its own instructions, its own narrow task, and no stake in the draft being good. A separate pass with a separate instruction produces an actual second opinion. Module 2.02 builds one of these, a critic that scores a draft as a number a loop can gate on, and later in Part 2 you will wire that same critic into a team of agents defined in one file.

The review agent closes one channel and leaves another open, and you should carry both. It catches drafts that break a rule you wrote down. It shares the weights’ blind spots with the generator, so a fact both models were trained to believe passes review, and a bias in the weights is not caught by a second copy of the same weights. That is why the human stays in the loop as an auditor of samples even when the agent does the volume.

what you can do with this today

You now hold the working picture: text becomes tokens, the network guesses one token at a time from a fixed set of weights, everything it can use has to be inside the context window, and nothing survives the conversation. Three failure classes follow from the absent certainty score, and every check on them sits outside the model, in one of the two loops.

The picture is a diagnostic you can run on any wrong answer, and it costs nothing. Ask three questions in order.

Was the material that determines the right answer in the window? Before rewriting a prompt, check whether it was actually present, and whether it was still inside the window when the answer was written. Most of the time it was not, and no amount of rephrasing will conjure it. If it was missing, that is a grounding problem or a context overflow, and the fix is to put the fact in and keep it in.

Did the answer sound sure? Then treat the confidence as carrying no information. Fluency and hedging are both generated by the same process as everything else. A model that sounds certain and a model that sounds cautious are telling you about the shape of the text, not about the reliability of the claim. Follow the citation; run the code; check the figure against the source.

Where was the check, and could the draft have talked it out of the verdict? Put your check where it cannot be: a citation you follow, a schema that validates, a test that runs, a second agent with its own instruction, anything that does not depend on the same pass that produced the draft. And if the wrong answer was a pattern across many outputs rather than a single slip, that is bias, and the check is a measurement across groups rather than a read of one transcript.

Here is the whole pass on one card, in the vocabulary you now own:

1. TEXT IN
   Your message, plus everything already in the conversation.
   All of it has to fit the context window; under a rolling history,
   older tokens drop out of view without warning.

2. TOKENS
   A fixed lookup table cuts the text into chunks. About 100 tokens
   per 75 English words. The model reads these, never letters.

3. THE PASS
   The weights, fixed and shared, read the whole sequence and score
   every candidate token. The model holds no internal certainty score
   for what is fact vs. plausible continuation.

4. ONE TOKEN OUT
   Scores become probabilities; one token is drawn; it is appended,
   and the pass runs again. A 200-word answer is ~300 passes.

5. THE THREE FAILURES
   Confabulation: a gap filled with the shape of an answer.
   Grounding gap: consistent with descriptions, wrong about the world.
   Systemic bias: the skew of the training text, continued.

6. THE TWO LOOPS
   You, at the input stage and the verification stage.
   A review agent, with its own instruction and no stake in the draft.
   Both sit outside the model, because nothing inside can look.

Keep the fundamentals to hand as a one-page field card: llm-reality-checklist.md.

Next we make the engine yours. Everything the model is lives in a fixed file of numbers, and in the next module you place that file on your own machine and run it: no account, no meter, and no data leaving your desk. The system built around the engine — the instruction that gives it a goal, boundaries, and contact with your data, committing it to one purpose; the word for what that produces is an agent — is the craft of Part 2. The syllabus points the way.