Separating claim gathering from judgment

Ask a language model what happened in the world this week, and it will answer fluently whether it knows the truth or not. In the foundational module on language models, we examined the mechanism behind this behavior: the underlying system optimizes for the plausible shape of a correct answer. A confident paragraph describing a complex negotiation carries the same structural shape whether the event actually occurred or was merely reconstructed from training data that ended months ago. For routine questions, this tendency is a minor nuisance; for live events (such as breaking election returns, a disputed labor strike, or a fast-moving rumor in your feed), it is the central failure mode, because the reader cannot distinguish the two kinds of prose either.

The solution is an architecture that isolates the gathering of claims from the drafting of the final synthesis. The agent drafting the synthesis holds no search tools of its own; it receives two reports prepared by collectors, and its instructions require it to draw only from them and to cite the specific source responsible for every assertion. Withholding the search tools limits what the writer can fetch. The model can still write a fact, a handle or a domain from its weights, so keeping the answer inside the two reports rests on the instruction, and on a check in code where a builder adds one. In this module, we examine this architecture using a production specimen: a fact-checking team of three agents, configured as an Arbiter for global news.

One statement about this module’s specimen and pictures, made once. The team answers on demand, one question at a time. The flow figure later in the module draws the corroboration gate as a stage the text passes through, where in the running team it is a rule inside the Arbiter’s instruction applied claim by claim as it writes. The evidence boundary and the consultation order are likewise instructions, and the corroboration labels are judgments the collector and Arbiter models write under instructions that do not ask them to trace a claim to its origin; no code in the specimen checks the answer against the reports, checks a report against its tool results, or counts sources. The trace in the run section is reconstructed so the orchestration reads clearly, while its facts, tool calls, and cited sources are the real ones.

You already hold every component needed to understand this design. When we studied delegation earlier in the course, we introduced it through mandatory consultation contracts. In our visual auditing pipeline, we demonstrated why gathering evidence must remain separate from judging it. Here, we apply those same structural separations to breaking news, governed by the corroboration gate.

The Riverside Repair Café, a fictional community event this course uses for practice, has two announcements of its autumn session:

Excerpt A, the organisers’ newsletter:

Riverside Repair Café, autumn session: Saturday 18 October, 10 a.m. to 1 p.m., Library Hall, Mill Street. Entry is free; bring one broken item per person.

Excerpt B, a neighbour’s post:

Repair café is back! “Saturday 25 October, Library Hall,” says the organisers’ newsletter. Bring your broken toasters. (@millstreet_mum)

two collectors and one arbiter separate gathering from judging

The team consists of three agents: two collectors and one arbiter. The power of this design lies in the capabilities we deliberately withhold from each role.

XResearcher monitors X (Twitter). Its configuration specifies gemini-3.8-flash with two tools. The first, x_api_search, is a client-side function that calls X’s own search API directly with a read-only X bearer token; because the search runs in the tool’s own code, the researcher can sit on any model provider. Every photo attached to a matching post arrives already transcribed beneath it, so a chart, a screenshot, or a headline reaches the report as the image’s own word. The second tool, web_extract, lets the researcher read an article a post links when the post alone does not explain itself. The researcher’s contract requires it to test at least three query variations, and caps it at eight searches, before reaching conclusions. It examines official statements and wire reports first, deliberately samples prominent accounts across competing viewpoints, and identifies both points of dispute and patterns of amplification. It produces a plain-text report containing distinct sections: numbered claims accompanied by user handles and corroboration flags, a breakdown of opposing perspectives showing what each side highlights or omits, prevailing public reactions, and identified information gaps. Crucially, its contract strictly forbids analysis. The agent gathers and catalogs evidence; it never interprets.

WebResearcher is the collector that reads the documented record. Running on gemini-3.8-flash with access to web_search and web_extract, it ingests two inputs: the original user query and the specific claims surfaced by XResearcher. Its operating procedure requires reading before ruling: it queries each claim, visits the most authoritative primary sources, and parses the raw page content under a strict extraction budget to prevent endless browsing loops. Its report establishes the documented record: verified facts categorized as CONFIRMED (two or more independent sources, or a primary source) or REPORTED (fewer), a point-by-point verdict for each handed-off claim (confirmed, contradicted, or unverified), and an explicit list of domains consulted. Whether two of those sources are independent is WebResearcher’s own judgment. Like its peer, WebResearcher is strictly barred from offering commentary.

The Arbiter drafts the final response, operating entirely without tools. It cannot query X, search the open web, or retrieve external pages. For every substantive query, its instructions mandate an ordered two-step consultation: it must invoke X first to capture emerging live claims, then query the web to verify those specific claims against authoritative records. The instruction lets it draft a sentence only after both structured reports are returned. Even if the underlying model believes it already knows the answer, it must execute both queries. The rationale is straightforward: model weights reflect past training data, whereas user inquiries concern ongoing developments in the present.

The evidence boundary: the answer is built only from what the two reports contain. A tool-free arbiter cannot slip its own searching into the answer; whatever it states, the instruction requires one of the two reports to carry, and a check in code is what would confirm that one does.

Set beside the visual inspection pipeline, this structure reads as a foundational design pattern rather than an isolated technique. There, the observer inspected only the visual elements while the auditor evaluated only the textual criteria, preventing expectations from distorting perception. Here, the collectors gather data without judging its merit, while the arbiter evaluates the evidence without performing collection. In both cases, isolating gathering from grading ensures that no single context window is ever burdened with (or compromised by) attempting both tasks at once. Practical experience reveals an operational reality: an agent actively parsing live, unstructured search results often struggles to maintain a strict formatting contract in the same execution step. Dividing retrieval and formatting across dedicated agents allows each model to execute its specific role with high fidelity.

two channels: the live claim stream and the documented record

A rumor appears in your feed now. The wire services and institutional releases that could confirm or deny it lag by hours or days. A checker that reads only the live claim stream amplifies the rumor. A checker that reads only the documented record cannot see the claim you asked about, because formal institutions have not yet acknowledged it. What would the system have to read to evaluate the event?

The architecture answers by reading both sources through two separate channels. The X channel carries the live claim stream: the assertions, framings, and reactions circulating across the network in real time. The web channel carries the documented record: wire reports, institutional releases, formal documentation, and published datasets.

The Arbiter calls the two collectors in sequence, so the live claims are in hand before WebResearcher inspects the formal documentation. XResearcher sweeps the live claim stream and generates a report of numbered claims, source handles, and disputed points. WebResearcher receives those claims as hypotheses to test and queries the documented record, producing a verification report that evaluates each assertion as confirmed, contradicted, or unverified. Because XResearcher’s claims shape WebResearcher’s searches, the two reports are coupled from the start: the second collector looks where the first one pointed. A channel is also a route, and two routes can carry one origin: a post that links an article and the article itself reach the Arbiter as two reports of the same press release. When findings from both channels align, that agreement carries weight only if each rests on its own originating evidence, the document, data, or first-hand report a claim traces back to. A claim whose trail ends at one origin has one source however many outlets and channels repeat it, and the last bullet of the trace below makes exactly that count. While multi-modal engineering typically refers to processing distinct formats such as images and text, this architecture applies a similar discipline to two text channels delivering distinct forms of evidence.

your turn: count the originating sources behind the reports

Count the café’s two announcements this way. The post by @millstreet_mum reaches the X channel; the organisers’ newsletter reaches the web channel. The post names the newsletter as its authority. If the post had quoted the text correctly, the post and the newsletter would represent a single origin observed across two separate paths.

The post gives another date, so the two carry different claims. The post is one source for its own claim, 25 October. The newsletter is the originating evidence for 18 October. Each claim is counted back to the document it started from, whatever channel carried it.