Multi-modal agents: predictable images, objective eyes

Vision models generate and read images, and they fail the way module 1.01 taught you to expect: the output is shaped like a correct answer whether or not it is one. A vision model miscounts objects, drops a required element, or changes a requested style, and the result looks entirely plausible, because plausibility is what the machinery optimizes for, in pixels as much as in prose. Nothing in the picture announces the miss, so the check has to come from outside the model that drew it.

Everything we build here is one arrangement for supplying that check, and you can hold it in a sentence before we build it. Decide in writing what the image must contain before anything renders; then split the review into two eyes that never share information, one that looks at the image without knowing what was asked for, and one that reads what the first eye wrote and compares it against the contract without ever seeing the pixels. Spec first, then a blind inventory, then a text-only audit. We call the second half the division of visual labor, and by the end you will be able to say, for any review of any image, which of the two eyes was doing the work and what it had been told before it looked.

We work through it on one specimen, the run that produced the image you will inventory yourself later in the module: a request for three phosphor-green amphorae on a stone shelf against a dark virtual terminal space, sent through melchizedek’s image production syndicate. An amphora is the tall, two-handled clay jar the ancient Mediterranean shipped its wine and oil in, and the only thing about it that matters here is that the request asks for exactly three, because a count is a claim an image can be held to. The syndicate is one definition file, config/agents/examples/image_production.yaml, and it fits the shape you have used all through Part 2: an orchestrator holding two tools and one sub-agent, each with its own instruction and its own clearance. What is new is which information each of them is allowed to hold.

the pipeline, end to end

Here is the whole path first, so that every part we take apart afterwards has somewhere to attach.

A request arrives: someone wants an image. The agent generates nothing. It writes a specification first, a structured JSON document naming the exact subject counts, lighting, palette, background, composition, aspect ratio, and the master generation prompt. Then it stops and waits for a human to approve it.

On approval, and only then, the agent calls generate_image. An image file exists.

Now the review, and this is the part with the real engineering in it. The image goes to a blind observer, a vision model given the image file and nothing else. No specification and no original request enter its window, so no token in it conditions the output toward an expected answer. Its job is to catalog what is physically present: subjects, counts, colors, background, lighting, artifacts. It produces a plain text inventory and makes no judgments.

That inventory, as text, goes to a conformance auditor, a text-only agent that never sees the image at all. It reads two documents, the approved specification and the blind inventory, and compares them field by field. Out comes a per-field verdict, a score, and specific corrective feedback.

Two models looked at this problem and neither one held both halves of the information. That is the design. Why it has to be split exactly this way is what we take apart next.

phase one: decide in writing

Why write a specification at all, when the request is already a sentence? Because a loose sentence cannot fail. Generate directly from “make it look professional” and the render is unpredictable, and worse, you have nothing to check it against, because nobody wrote down what passing would mean. A review needs a contract, and the contract has to exist before the thing it grades.

Look at what the designer wrote for the amphorae before it rendered anything. The trace below shows the payload excerpted to its load-bearing claims: primary_focus says exactly three ancient Greek amphorae, evenly spaced; location says a virtual dark terminal space; medium says a CRT vector-graphics render; color_palette says monochromatic phosphor green on pure black; aspect_ratio says 16:9. Five fields, and every one of them is a claim the finished image can be held to. “Evenly spaced” can be checked; “nice” cannot.

So the visual designer agent runs in two phases, and the boundary between them is enforced in its instruction:

# PHASE 1: SPECIFICATION & CONFIRMATION
# Generate the JSON specification payload and wait for explicit approval.
# NEVER invoke generate_image during Phase 1.

# PHASE 2: GENERATION
# Execute generate_image only after receiving human confirmation.

Read that as: the agent has two states, and the tool that makes pixels is forbidden in the first one. Nothing renders until a person has read the fields and said yes.

That is the last line of the reasoning, so take it as the rule it is:

Write the contract before anything renders. Define visual requirements in a structured schema before the first render. Verification requires an explicit contract to compare against; a field that was never written down can neither match nor fail.

Two earlier mechanisms are doing the work here. The two-phase instruction is module 2.01’s workflow-stages block, the section of the written anatomy that fixes the order an agent must follow across several turns; here it fixes that generation comes second. And the human approval is module 1.01’s input stage, where you decide what enters the window before anything is produced: the cheapest place to fix an output problem is before generation, where changing your mind costs a sentence rather than a render.

The habit at this scale is small. Before you ask for an image, write the fields you would grade it on. If a field will not go into words, you have not decided it yet, and it is the field that will come back wrong.

why the generator cannot grade itself

Now the review, and first the reason the reviewer cannot be the maker. Module 2.02 gave you the rule and the mechanical reason behind it: everything the drafter produced is sitting in its own context window, so when it is asked “is this correct?” agreement is the statistically likely continuation, and you get a fluent explanation of why the draft is fine. The maker must never grade the making.

Vision adds a second, sharper failure, and this one catches people who have already accepted the first.

Suppose you avoid the obvious mistake and hand the image to a different model for review. But you also hand it the specification, because how else would it know what to check? Now the reviewer has been told to look for three green vases. It looks at an image containing two, and reports three.

That is expectation priming, and human perception shows the same failure. A human proofreader reads straight past a typo because the brain supplies the word it expects; a radiologist told what to look for finds it more often than the film supports. Telling a reviewer the answer contaminates the review. And notice that this is the same property that makes a worked example in a prompt effective: what the model was shown first shapes what it is most likely to produce next. The mechanism is doing exactly what it always does; the reviewer’s chair is the one place you cannot afford it.

So put the two findings together. The reviewer must not be the maker, and the reviewer must not know the target. But something has to know the target, or there is nothing to grade against. The only arrangement that satisfies both is to give the looking and the judging to two different minds: one that sees the image and has never read the specification, and one that reads the specification and the first mind’s report and has never seen the image. Each holds exactly one half. That is the last line of the reasoning, and it is the rule everything we build from here runs on:

The observer sees only the image; the auditor only text. Perception and judgment are separate agents with separate clearances. The observer sees the image and nothing else, so it cannot be primed. The auditor sees the specification and the observer’s inventory and never the image, so it can only judge recorded observation. The maker approves the spec and decides what to do with the verdict, and never grades its own work.

The pipeline splits inspection into two isolated steps, and the isolation is the whole product. Now the two agents that hold the two halves.

the blind observer

The inspection tool, inspect_image, takes exactly one parameter: a local image file path. It has no access to the specification and no access to the original request, because those parameters do not exist on it. Read that as: the blindness is structural. A designer who wanted to hand the observer the spec could not, because there is no slot to put it in. An instruction that says “please be objective” is one more stretch of text, weighed like any other text in the window; the missing parameter blocks the leak outright.

It runs a fixed unprimed protocol against a vision model, cataloging what is visible: subject elements, object counts, dominant colors, background structures, lighting, surface artifacts. It outputs a plain text inventory of observations and passes no judgment on whether any of it is correct, because its window holds no contract to judge against. The protocol’s rules are worth reading for what they refuse: counts are exact numbers, never “several”; no quality words; no guesses about intent. Anything the observer is unsure of is written as uncertain, with what is visible described.

Why is refusing to judge what makes the inventory more useful, and not less? Because the auditor can only compare observations. “Three amphorae, evenly spaced, centered on a horizontal shelf” can be set beside a spec field and matched or failed. “A beautifully balanced composition of three elegant vases” cannot; the adjectives are verdicts, and a verdict cannot be set beside a spec field and compared.

the conformance auditor

The SpecAuditor is a text-only leaf agent with a structured output schema. It has no vision tools and never sees the image.

Recall the constraint from module 2.04: an agent with an outputSchema must be a leaf, with no delegation powers, or the engine deadlocks. That is why the auditor sits at the edge of this graph. Its structured output is what turns a visual opinion into something a loop can gate on, exactly as module 2.02’s critic turned quality into a number: a gate reads a machine-parsed score, never conversational text, and a conformance integer is a score an if statement can read.

It reads the approved specification and the blind inventory, compares field by field, and returns:

• per-field status: match, mismatch, or unverifiable
• a conformance score from 0 to 100
• specific corrective feedback when the score falls below threshold

Read that as: one line per spec field, each naming what was expected and what was observed and the verdict between them; the score is the share of checkable fields that matched; and the feedback names the one field to reinforce if the image has to be made again. Every verdict traces to a quotable spec line and a quotable inventory line, which is what makes a wrong audit debuggable: the observation was bad, or the comparison was.

The unverifiable verdict is the one people leave out, and it matters. If the specification demands a particular lighting temperature and the inventory says nothing about lighting, the honest answer is neither pass nor fail. Collapsing that into “match” is how audits become theatre. The auditor’s own rule is the plain version: the inventory is its only source of observations, and if the inventory is silent on a field the verdict is unverifiable, never a guess.

a real failure, caught

Now the amphorae run itself, in a live production session, against the contract we read in phase one: three phosphor-green amphorae resting on a stone shelf against a dark virtual terminal space.

The designer wrote the specification, received approval, and called generate_image. Then the review ran:

The audit came back at 57 out of 100. Work the number, because it is checkable from the verdict line in the trace. The auditor returned eight fields. One, aspect_ratio, was unverifiable, because the inventory protocol reports no pixel dimensions, so seven were checkable. Four matched: the primary focus, the foreground, the perspective, the framing. Three did not: the location, the medium, the color palette. Four of seven is 57 percent, and that is the whole arithmetic. The count of amphorae survived; the medium failed. The generating model had produced a photograph of a physical CRT monitor displaying the graphic, rather than the raw digital graphic itself, and the trace’s own verdict ties two of the three mismatches to that one event: the medium, because a photograph of a monitor is not the vector render that was asked for, and the location, because the blind eye reported dials and cables where the spec promised a virtual dark terminal space. The auditor marked the palette as well; the verdict does not spell that one out. The likely reading, that a photographed monitor is no longer monochrome on pure black, is ours, not the trace’s.

Notice why the division caught it. The blind observer had no expectation of a monitor, so it reported what it saw: physical dials, glass reflections, a plastic housing, background cables. A primed reviewer, told to check for amphorae on a shelf, would very likely have confirmed amphorae on a shelf, because they were there, inside the photograph of a screen. The text-only auditor compared “plastic monitor housing, glass reflections” against a specification demanding a virtual background, and flagged the mismatch. Neither of them knew what the other knew, and the designer, which had approved the spec, was never asked to grade the result.

The failure was in a field nobody would have thought to check, which is precisely the kind an unprimed observer finds and a primed one does not.

Grade the instrument on its own weak point while it is in front of you. The unverifiable on aspect ratio is a real gap, and the run tells you where it is: the observer’s protocol had no line for dimensions, so the judge had nothing to read. The fix follows from the division law. You widen the witness by adding a dimensions line to the inventory protocol; you never let the judge peek at the image to settle it. Every unverifiable verdict is a to-do for the observer’s protocol, and none of them is a pass.

Now take the observer’s chair. The image below is the real output from that run. Inventory it before the specification is revealed, then compare your eye against the actual audit:

┌─ bench · you are the blind inventory ─ exercise

equip yourself with the materials

1. The complete image production syndicate: image-production-syndicate.md.
2. The portable inventory protocol and audit schema: visual-inventory-protocol.md.
3. Run the pipeline: npm run syndicate:image (Node 22+ and a Gemini key; this module’s generation model wants a paid tier).

what you can do with this today

The division of visual labor is a set of questions you can ask on a working day, about any review, of images or of anything else.

Was the contract written before the render? If the answer is no, write it now, as fields, before you look at the output again. Anything you cannot state as a checkable field is something you have not decided, and it is the field that will come back wrong. That is the contract, written first, and it costs a sentence.

What did the reviewer know before it looked? If it had seen the target, its report is a primed report, and a match tells you less than it seems to. Ask what the reviewer knew that could contaminate the review, and remove it if you can. This generalizes well beyond images: any time you ask a model to check work, the same question applies.

Can the judge see the thing it is judging? If it can, the thing enters its window and conditions the verdict. Make the judge work from the observer’s transcript, so every verdict traces to two quotable lines and a wrong verdict can be sent back to whichever of the two was at fault.

Does the verdict have a third value? A check that can only pass or fail will quietly turn absences into passes. Keep unverifiable, and treat each one as an instruction to widen the observer’s protocol.

Here is the whole loop on one card, in the vocabulary you now own:

1. SPECIFY        the maker writes the fields; a human approves; nothing renders
2. GENERATE       generate_image runs, once, after approval
3. OBSERVE        inspect_image(path) — the observer sees the file and nothing else
                  → a plain inventory: exact counts, no judgments, no guesses at intent
4. AUDIT          SpecAuditor(spec + inventory) — text only; the image never enters
                  → one verdict per field: match · mismatch · unverifiable
                  → conformance = matching fields / checkable fields
                  → recommendation: accept, or regenerate naming ONE field
5. DECIDE         the maker reads the verdict and chooses; it never grades
                  unverifiable → widen the observer's protocol, never the judge's view

Next the same separation of channels is turned on facts themselves. In the coming module two researchers sweep different records of the same events, neither seeing the other’s work, and an arbiter that never collects weighs the two reports line against line — the blind observer’s discipline, applied to the news. Open Module 2.08 to continue.