An independent critic scores each draft

the critic loop scores quality as a number

Manual debugging builds rigorous engineering intuition. It cannot scale to production workloads processing thousands of requests per hour where human inspection on every turn is impossible.

To support production scale, we embed quality verification directly into the system topology. We call that topology the critic loop: a secondary model positioned between the primary generator and the end user, tasked solely with quality verification and reading the draft in a context window separate from the one that wrote it. In the critic configuration this module quotes, this workflow is organized as a bounded improvement loop partitioned into two roles: the drafter produces the draft response, and an independent critic scores that draft against the quality bar. A draft below the bar goes back to the drafter for at most three rounds, and the turn returns the critic’s latest verdict with its score attached.

Rather than returning free-form prose commentary, the critic emits a strictly structured schema:

- name: "CriticAgent"
  output:
    schema:
      type: "OBJECT"
      properties:
        message:    { type: "STRING" }
        confidence: { type: "INTEGER" }   # below 85 triggers a revision loop
        issues:     { type: "ARRAY", items: { type: "STRING" } }
      required: ["message", "confidence", "issues"]
    mime: "application/json"

Read that as: the critic is permitted to answer with only three fields, a string, an integer, and a list of strings, and the orchestrator’s instruction branches on that integer. Each field has one reader: the user reads message, the orchestrator reads confidence, and on a failed round the drafter reads issues, one entry per defect, quoting the words at fault. The integer confidence is a rubric score: the critic model assigns it by following the scale its own instruction defines, and nothing calibrates that scale against measured correctness, so a score of 85 records that the critic placed the draft in its top band. It is not an 85 percent probability that the draft is correct; module 2.13 shows how to measure a judge’s scores against human labels. In the full configuration, the critic’s own instruction defines the integer’s scale, cut at the gate: 85 to 100 means every claim checks out as far as the critic can tell, the question is fully answered, and the draft ships; 60 to 84 means fixable gaps, such as an unanswered part or a claim the critic cannot check; below 60 means a factual error, a misleading claim, or most of the question unanswered. Because the top band starts at 85, the scale’s word for a passing draft and the gate’s threshold are the same line, and the instruction forbids a score of 85 or above while issues still holds an entry. Pinning the reply to named keys turns the critic’s judgment into a value that an orchestrator, or code, can compare against a threshold. That reasoning establishes our rule:

A gate compares a parsed score against a threshold: an automated quality gate reads one machine-parsed field and leaves conversational text, such as a paragraph saying “this looks pretty good”, out of the decision. A comparison written in code returns the same branch for the same score every time; the score it compares remains the critic’s judgment.

This applies the verification principle behind every written rule: a rule that can be checked as a pass or a fail against an execution transcript is a rule you can evaluate dependably. Software cannot reliably parse conversational evaluations such as “is this response good enough?”, and it can compare an integer. In this configuration no code makes that comparison. The ReviewOrchestrator is itself a language model, and its instruction tells it to read only the confidence integer, compare it with 85, and either loop or return; a capable model usually follows that instruction, and nothing in the engine checks that it did. When your workflow must never release a draft below the bar, your application code reads the returned JSON and evaluates if confidence >= 85: itself, then withholds the draft, returns a failure, or routes it to a person when the check fails. That comparison is deterministic, and the integer it compares is still a judgment the critic wrote.

Whether a draft answers the question well is a judgment, and the critic’s instruction handles it the way our lesson on agent design handled any task that needs one. It names the decision, fixes the outcomes as three bands, defines each band by what the draft contains, and forbids a passing score while issues holds an entry. The measurable language surrounds the judgment, and the judgment itself stays with the model.

the critic must be an independent agent

It is natural to wonder whether we could simplify the architecture by asking the drafting model to critique its own response within the same ongoing conversation. However, the mechanics of a single context window work against it.

A cheaper review seems to be on hand: a drafter writes an answer, and in the same conversation you ask it, “is this answer correct?” What tends to come back?

When a model drafts a response, its context window fills with its own internal reasoning, working assumptions, and vocabulary selections. Those generated tokens exert strong statistical conditioning over any subsequent tokens generated in that window. If you ask “is this answer correct?” in that same context, the model tends to defend its prior output, as the preceding text stands as an extended argument for the conclusion it just reached. The prompt is likely to produce an articulate justification for the draft, because the exact context conditions that generated the error remain active in the window. The arithmetic beneath it is plain: self-review in the same pass runs identical weights over the very context that produced the original defect.

Isolating the review into an independent invocation provides three structural advantages:

An independent context window. The critic is handed the user’s question and the draft, and it reads them under its own instruction with the evaluation rubric. None of the drafter’s intermediate generation history conditions its token distributions, which removes the drafter’s own argument for its draft from the window that judges it and reduces the tendency to rationalize the intent. The critic’s window is separate from the drafter’s, and it persists across rounds: in this configuration the CriticAgent’s session is kept for the conversation, so on round 2 its window still holds the round-1 request with draft v1 and its own round-1 verdict, and a later question in the same conversation finds those earlier reviews there too.

A specialized instruction. The critic’s prompt focuses exclusively on quality assessment and defect detection, with no conflicting requirement to answer the user’s primary prompt. A model can verify a factual claim only against evidence it holds. The CriticAgent in this configuration holds no tools and receives the question and the draft and nothing else, so the facts it can check a draft against are those in its weights and in those two texts, and its rubric sends any claim it cannot check to issues as unverified: <the claim>. To have a critic verify factual claims, you pass it the source material, such as the reference document or the retrieved passages, alongside the question and the draft: the context variable from earlier in this module, applied to the reviewer.

Bounded revision routing. If the score falls below 85, the orchestrator, following its instruction, sends the drafter its previous draft with the critic’s issues quoted in full, for at most three rounds; after the cap it returns the critic’s latest verdict with its score, whatever that score is. The round cap is also an instruction to the orchestrator model. The engine’s own hard bound is max_steps, a limit on model calls per turn that defaults to 50 when, as in this configuration, the file sets none; it stops a runaway orchestrator long after three rounds would have ended.

The critic is also the course’s first case of a limit that no prompt edit reaches. The drafter’s defect lived in what its window held, its own draft and the reasoning behind it, and a better-worded review instruction leaves all of that in place. Refining one agent’s prompt changes what the model does with its context. When the defect is the context itself, you split the task and give the part that needs a different window an agent of its own. Module 1.09 turns that split into four topologies and says when each one pays for itself.

The source configuration also records a constraint that testing discovered. Under Google ADK, the runtime the configuration was first written for, an agent configured with an output schema could not also delegate work to peers: an agent holding both stalled indefinitely. The configuration therefore puts the schema on a leaf agent and gives a separate orchestrator the sequencing. A later runtime lets an orchestrator delegate first and then answer in its own schema. The critic keeps its leaf by choice: a grader that holds no tools is handed the question and the draft and fetches nothing else, which is the independent context window this section argues for.

Be mindful of the gate’s inherent boundaries as well. If both the drafter and the critic in this configuration rely on the same underlying model, any factual omission or blind spot present in the base model weights will exist in both context windows. A separate critic call isolates context biases, but it cannot repair missing knowledge in the base weights. The gate depends on the critic’s calibration: the threshold of 85 evaluates a self-reported score. What the design closes is the same-window agreement failure, and that is the one it was built to close.

An unedited production execution trace from 2026-07-11 records four runs of the loop. When prompted with a two-sentence inquiry into the decline of the Library of Alexandria, the system satisfied the rubric in a single pass with a confidence score of 98. When subjected to three targeted stress tests (a standard inquiry, an inquiry with a false premise about the Antikythera mechanism and steam, and a request to list exactly twelve primes strictly between 100 and 150) each draft satisfied the rubric in a single pass: the drafter corrected the false premise, and for the impossible request it gave the count, ten, that shows the request cannot be met. These four runs establish that those four drafts cleared the bar on their first review. They do not measure how often the gate catches a failure, and the gate exists for the drafts that do not clear it.

The returned JSON is also how you tell the outcomes of a turn apart, and in each case the code settles part of what happens and the orchestrator’s instruction settles the rest. When the critic scores 85 or above on the first review, or on a later round after a revision, the orchestrator returns the critic’s JSON: message, a passing confidence, and an empty issues list. When the draft is still below 85 after the round cap, the instruction tells the orchestrator to return the latest JSON regardless of confidence, so you receive a score below 85 with a non-empty issues list, and no code marks that turn as a failure; the application-code check described above is what would. When the critic’s reply is malformed, the engine has already asked Gemini for JSON under the schema, and it parses the reply: text that does not parse fails the tool call, and the orchestrator receives the parser’s error or, under the engine’s default self-correction, retry guidance in its place. The parse checks syntax only, so valid JSON that lacks a field passes it. The retry that follows is the orchestrator’s guardrail, which sends the critic the same question and draft once more and counts that retry toward the round cap.

When the critic’s provider is unavailable, the engine retries transient failures, three attempts in total by default, and a call that still fails ends the CriticAgent’s run and returns an empty result to the orchestrator. What the orchestrator then does with an empty result, and what you see, no run in this course has recorded. In every one of these cases the gate and the round cap are executed by the orchestrator model, and the only bound on the loop written in code is max_steps.

A run of the loop returns the critic’s JSON, so you read each verdict by its fields. An ordinary question shows the loop’s simplest path, and in the recorded trace each such draft passed on its first review. A false-premise question, such as one asking how steam engines powered the Antikythera mechanism, tests two things at once: whether the drafter corrects the premise, and whether the critic’s confidence and issues agree with the band the draft falls in. A score of 85 or above with an entry in issues breaks the critic’s own instruction, and you can spot it in the JSON without rereading the draft.

what you can do with this today

Every concept in this module translates into concrete diagnostic questions you can apply to active projects immediately:

What, in one sentence, is wrong with this output? Articulate this diagnosis before editing your prompt. That single sentence identifies the target variable. Modifying a prompt before you can name the defect is merely guessing.

Which one variable does that sentence name? Adjust that isolated variable and re-evaluate on the failing input. Revert any edit that fails to improve the result so your prompt remains clean and legible.

Did I keep the input that broke it? Transform every resolved failure into a permanent test case. Retaining failing inputs creates the most reliable and inexpensive regression suite available in prompt engineering.

Who is grading this, and were they in the window that wrote it? Never permit the drafting pass to grade its own output. Whether you perform the review manually during development or delegate it to an automated critic in production, quality assessment must execute in an independent context window, and automated gates must evaluate a machine-parsed score rather than free-form text. A gate that must hold is a comparison your own code makes on the returned score.

Use this operational summary as a quick-reference guide for your debugging workflow:

BY HAND — one variable at a time
1. RUN      the agent on a real input, and the awkward ones
2. NAME     one sentence: what is wrong with the output
3. PICK     the variable that sentence names
              clarity     it did the wrong task
              precision   right task, wrong shape or size
              context     a confident invention
              instruction it described instead of did
              format      correct but unparseable
4. CHANGE   that one variable, nothing else
5. RE-RUN   same input; revert what did not help
6. KEEP     the failing input as a test

IN THE LOOP — the critic's file
1. DRAFTER  produces the response, its own window
2. CRITIC   a separate call, a leaf with the schema:
              { message: STRING, confidence: INTEGER 0–100,
                issues: [STRING] }
3. GATE     the orchestrator reads confidence:
              >= 85 ships; below 85 loops with the issues
4. CAP      three rounds, then the latest verdict ships, score attached

Distributing work between a drafting specialist and an independent critic represents the foundational pattern of multi-agent engineering. In our next lesson, we will expand these topologies: examining four distinct patterns for agent coordination, treating orchestrator instructions as rigorous operational handoff contracts, and configuring distinct agent voices using countable system laws—all running locally on open weights. Consult the syllabus to continue.