Grading an image without priming the reviewer

neither the designer nor a primed reviewer can grade the image

Suppose you delegate the review of an image to an independent vision model, and you place the specification in its prompt so it knows what to verify. If the specification asks for three green vases, the model will frequently inspect an image containing only two and report three.

The reviewer miscounts because earlier context tokens condition subsequent token probabilities. When you instruct the vision model to search for three green vases, the tokens of that instruction alter the output logits before the model finishes processing the image tokens, biasing generation toward confirming the prompt. This vulnerability mirrors familiar patterns in human cognition: a proofreader skims past typographical errors because the brain anticipates the intended word, and a radiologist informed of a suspected condition detects that pattern in a scan more frequently than the scan alone justifies. In multimodal systems, this behavior is known as expectation priming.

The conditioning that allows few-shot prompt examples to steer generation is the exact mechanism that blinds a primed reviewer: earlier context tokens alter subsequent token probabilities whether those tokens were written to demonstrate a pattern or merely state a requirement.

Self-evaluation by the designer represents this same conditioning in an even tighter loop. When the model that generated the image evaluates its own work, its context window still contains its drafting steps and design rationale. Asking the designer whether its output is correct pushes token probabilities toward self-confirmation, producing a fluent justification of its own mistakes.

Neither the designer nor a primed reviewer can provide an objective evaluation. At the same time, verification requires checking the generated image against the specification. To resolve this tension, we divide perception and judgment into two separate roles: a blind observer that inspects the image without knowledge of the specification, and a conformance auditor that checks the blind observer’s textual inventory against the specification without ever viewing the image file. Each component holds only half of the context. This structure forms our governing principle:

The observer sees only the image; the auditor only text. Perception and judgment are separate agents with separate clearances. The observer sees the image and a fixed inventory protocol, so it cannot be primed by the specification. The auditor sees the specification and the observer’s inventory and never the image, so it can only judge recorded observation. The maker drafts the spec, relays the verdict with its recommendation, and never grades its own work.

By separating inspection from evaluation into two isolated stages, we keep the specification out of the observer’s context and the image out of the auditor’s. We turn first to how the blind observer converts an image into an unprimed, factual inventory.

the blind observer produces a factual inventory

The inspection tool, inspect_image, accepts only a single parameter: a local image file path. It contains no parameters for prompt text or specification data. This isolation is structural rather than purely prompt-based; because the tool schema provides no field for specification inputs, context leakage is physically prevented at the API boundary. Prompt instructions asking a model to “remain neutral” can still be diluted within a context window, whereas omitted parameters establish a hard information barrier.

The observer executes a fixed inventory protocol across the image in six numbered sections: every subject with an exact count; the composition, meaning framing, perspective and placement across foreground, midground and background; the direction and quality of the light; the dominant colors and tonal range; medium cues that mark the image as photographic, painterly or rendered; and any lettering, watermarks or generation artifacts. Because its context window contains no target specification against which to grade, it describes visible features without evaluating compliance. The protocol sets its reporting rules: counts must be stated as exact numbers rather than indefinite quantities like “several,” quality words such as “beautiful” are forbidden, and guesses about intent or purpose are disallowed. When visual details are ambiguous, the observer is told to write “uncertain:” and describe what is visible. These rules are instructions to the vision model, and the inventory is that model’s description of the image. Keeping the specification out of its context protects against expectation priming; the model can still miss a feature or misdescribe one on its own, and no later stage looks at the image to catch it.

This strict refusal to judge is precisely what makes the inventory valuable for verification. A factual record stating “three amphorae, evenly spaced, centered on a horizontal shelf” can be directly evaluated against a specification field. Conversely, a subjective statement such as “a beautifully balanced composition of three elegant vases” provides evaluative commentary rather than checkable data, making structured verification impossible.

the conformance auditor gates on a schema

The SpecAuditor operates as a text-only leaf agent with a strictly defined output schema. It has no vision tools and never receives image inputs.

The auditor holds the schema because its verdict is what the review returns. An agent that declares an output.schema ends every turn on that JSON alone, while the designer’s own turns must stay free to return a payload, a file path or a relayed verdict as text. A team gives this placement to any specialist whose own answer is the result, and the auditor sits at that leaf. Its structured output puts the auditor’s comparison into typed JSON fields: a verdict string per field, a conformance integer, and a recommendation string. Code can read those values with an if statement, and that comparison is deterministic; the values themselves are the auditor model’s judgments, as a quality-scoring critic’s integer is, and the auditor can misjudge a comparison the way the observer can misdescribe an image. In this specimen no code reads them at all: the designer relays the verdict to you, and you decide.

The auditor ingests the approved specification alongside the blind inventory, conducts a field-by-field evaluation, and produces:

• a status verdict for each field: match, mismatch, or unverifiable
• an aggregate conformance score from 0 to 100
• a recommendation: accept when no material field mismatches, or regenerate naming the one spec field whose reinforcement would fix the most important mismatch

Read that as: one entry per audited spec field, each summarizing what the specification demanded and what the inventory reported, with the verdict between them; the score is the share of checkable fields that matched; and the recommendation names the one field to reinforce if the image has to be made again. The expected and observed entries are the auditor’s own brief summaries of the two documents, so when a verdict looks wrong you go back to the specification and the inventory themselves and find where the error entered, which is what makes a wrong audit debuggable: the observation was bad, or the comparison was.

A specification sets the lighting at 5600K. The observer’s inventory of the image says nothing about lighting. The auditor reads only these two texts, never the image.

The unverifiable status represents an essential safeguard in production audits. If a specification defines a precise lighting temperature but the blind inventory omits lighting details, the system cannot soundly report a pass or a failure. Treating omissions as automatic passes degrades auditing into superficial compliance. The auditor follows a clear operational rule: the inventory is its sole source of visual truth, and when the inventory is silent regarding a specified field, the verdict must be marked unverifiable rather than assumed.

The auditor’s instruction also sets the arithmetic: conformance is matching fields divided by checkable fields, so an unverifiable field leaves both the numerator and the denominator. That keeps an omission from counting as a match, and it also lets a high score rest on very little, because a score computed over checkable fields says how well the checked part matched and says nothing about how much was checked. You report coverage, the fields checked out of the fields the specification states, beside the score. You also mark which fields are required fields, and a required field that comes back unverifiable prevents accept or sends the image to a person for review. The specimen does neither. Its acceptance rule looks only at mismatches, so an unverifiable field never blocks accept, and the score and the recommendation are instructions to the auditor model that no code recomputes or acts on. A builder adds coverage and the required-field rule in code, counting against the specification’s own fields, because the auditor is told to cover the fields the inventory can speak to and a field can drop out of its list entirely.

a real failure, caught: the audit that scored 57

Let us examine how this architecture performed during an actual production run using the prompt defined in phase one: three phosphor-green amphorae resting on a stone shelf against a dark virtual terminal space.

The designer drafted the specification, received confirmation, and invoked generate_image. When the user then asked for a review, the designer ran phase 3.

The resulting audit returned a conformance score of 57 out of 100. Let us calculate that metric directly from the trace log. The auditor evaluated eight distinct fields. One field, aspect_ratio, was marked unverifiable because the standard inventory protocol does not extract explicit pixel dimensions; this left seven checkable fields. Among these, four fields matched: primary focus, foreground, perspective, and framing. Three fields failed: location, medium, and color palette. Four matches out of seven checkable fields yields 57 percent, and against the eight fields the audit listed, seven were checked. The subject count passed verification, but the medium failed completely. Instead of creating a direct vector-style digital render, the image generator produced a photograph of a physical CRT monitor displaying the artwork. The auditor’s verdict traced two of the three mismatches directly to this artifact: the medium failed because a photograph of physical hardware does not match a direct vector graphic, and the location failed because the observer cataloged physical knobs and external cables instead of an empty virtual terminal space. The auditor also flagged the color palette; while the trace does not elaborate, capturing an illuminated physical screen introduces ambient reflections and housing tones that deviate from a pure monochrome green palette on black.

This subtle discrepancy was caught specifically because of our decoupled architecture. Having no knowledge of the prompt, the blind observer recorded what was literally visible: physical dials, screen glare, monitor bezels, and cabling. Conversely, a primed reviewer instructed to look for amphorae on a shelf would easily confirm their presence inside the photographed monitor display. The text-only auditor compared observed physical monitor components against a specification requiring a purely virtual background and flagged the violation. Neither model possessed access to the other’s complete context, and the generating agent was never tasked with reviewing its own output.

This failure emerged within an environmental detail that human prompt engineers rarely consider checking: precisely the category of defect an unprimed observer can surface, as it did here.

The unverifiable verdict on aspect ratio highlights an operational gap: the observer’s protocol does not ask for image dimensions, and inspect_image does not read them, so the auditor lacked data to grade that field. The gap matters here because generate_image passes the requested ratio to the image model only as words in the prompt, and the saved file has whatever dimensions the model returned. Pixel dimensions are a property of the file: a PNG records its width and height in its IHDR header chunk, and a JPEG records them in its start-of-frame marker. Code that holds the file’s bytes, as inspect_image already does, can read them exactly without calling a model, whereas a vision model asked to report them would turn a measurement into a description. The shipped inspect_image does not extract them today; in your own pipeline, the tool reads the header in code and passes the dimensions to the auditor as plain text beside the inventory. The division of visual labor holds: code measures, the observer describes, and the auditor still never inspects the image. Every unverifiable verdict names a gap in what reached the auditor, and the fix goes upstream of it, into the protocol or into code.

The same formula shows why coverage belongs beside the score. As a hypothetical on this run’s eight audited fields: had six come back unverifiable and the remaining two matched, the formula gives 2 ÷ 2 = 100, with two of eight fields checked, and the acceptance rule as written, which looks only at mismatches, would allow accept. Coverage of two out of eight is the number that shows how little the 100 rests on.

what you can do with this today

You can apply the principles of the division of visual labor whenever you need to evaluate complex generative outputs reliably.

Was the contract written before the render? If no specification exists, write one immediately as structured fields before re-evaluating the output. Any requirement you cannot articulate as a testable field represents an unmanaged assumption, which is often where generative failures occur. Defining a structured contract beforehand requires minimal effort and prevents recurring defects.

What did the reviewer know before it looked? If the observer was given the target prompt or specification beforehand, its findings are primed, and positive matches become unreliable indicators of quality. Identify what contextual information could bias the reviewer, and remove it from the context window. This discipline applies to verifying code and text generation just as effectively as images.

Can the auditor see the thing it is judging? When the auditor directly views the raw artifact, surface details can distort its reasoning. Require the auditor to grade an unprimed inventory transcript instead; this links every verdict to specific reported evidence and allows you to debug whether an error originated during observation or comparison.

Does the verdict have a third value? Binary pass/fail systems frequently mask missing information by defaulting to positive assumptions. Always include an unverifiable status. When an unverifiable verdict reflects a visual feature nobody described, treat it as a reason to expand your inventory protocol; when it reflects a property of the file such as its dimensions, read that property in code. Report coverage beside the conformance score, and make a required field that comes back unverifiable block accept.

What happens to a required detail the observer cannot resolve? Specify a detail too small to resolve at the image’s size, such as a maker’s mark on one amphora, approve and generate, then run the review. Look at whether the verdict lists that field, how the score treats it, and what the recommendation says, and compare what you see with the coverage and required-field rule above.

Here is the complete operational workflow for reference:

1. SPECIFY        the designer writes the fields; a human approves (a prompted rule)
2. GENERATE       generate_image runs, once, after approval; require_approval would enforce it
3. OBSERVE (on request)
                  inspect_image(path) — the observer sees the file and a fixed protocol
                  → a plain inventory: exact counts, no judgments, no guesses at intent
4. AUDIT          SpecAuditor(spec + inventory) — text only; the image never enters
                  → one verdict per field: match · mismatch · unverifiable
                  → conformance = matching fields / checkable fields
                  → recommendation: accept, or regenerate naming ONE field
                  → add in code: coverage = checked fields / specified fields
                  → add in code: a required field left unverifiable blocks accept
5. DECIDE         the designer relays the verdict unedited and recommends; you choose
                  the designer never grades
                  unverifiable → measure in code or widen the observer, never the auditor's view

In the next module, we apply this same separation of gathering from judging to factual verification. Two research agents collect evidence through separate channels, one reading X and one reading the web, and the arbiter hands the claims the X researcher surfaced to the web researcher as things to verify. The arbiter itself holds no search tool, and it reconciles the two reports line by line, which applies the auditor’s discipline to claims about the world. Proceed to multi-agent fact-checking to continue.