Testing & refining an agentic workflow
by the end you can:
- Diagnose failures using the five variables: clarity, precision, context, instruction, and format
- Design adversarial-persona test scenarios to intercept failures before deployment
- Encode a strict quality bar as structured, machine-checkable output
- Transform a resolved failure into a permanent regression test case
When an agent produces a failing output, the underlying model is rarely broken. The failure almost universally resides within the prompt, specifically manifesting across one of five variables. Prompt engineering is never about phrasing a wish; it requires a rigorous, iterative dialogue: execute the run, diagnose the failure, isolate and adjust one variable, and execute again. This module teaches manual diagnosis before revealing the architectural endgame: Melchizedek’s critic syndicate. In this syndicate, autonomous agents perform the entire run-diagnose-adjust loop on each other, gated by a single parsed integer.
the five debug variables
When an output fails, systematic adjustment of these variables—one at a time—must precede any assumption of model failure:
- Clarity: Eliminate ambiguous jargon and tangled syntax.
- Precision: Establish hard boundaries, exact counts, explicit scope, and definitive tone.
- Context: Provide the necessary background information the task silently assumed.
- Instruction: Utilize strong, unambiguous cognitive verbs (e.g., synthesize, contrast, extract).
- Format: Demand the exact output structure as a rigid contract.
The discipline demands diagnosis first. Each class of failure points directly to a specific variable. Adjusting the wrong variable often exacerbates the problem; generating a structurally tidier hallucination does not cure the hallucination. Prove this process yourself. Every output in the panel below derives from a live run:
Before automating the process, deploy one final manual instrument: the adversarial persona. Instructing the model to “Act as a highly skeptical QA analyst” redirects the machinery that generates confident answers into a mechanism that generates confident, systemic attacks. It actively seeks ambiguous inputs, false premises, and missing-context traps, ranking them by probability and potential harm. Execute this persona against your agent’s core instruction before exposing it to users. When it accurately predicts a failure, file it immediately as a permanent regression case, preserving the input, the failed output, and the validated fix. Retrieve the ready-to-use persona here: qa-persona-prompt.md.
the loop that grades itself
Manual review cannot scale beyond isolated testing. Melchizedek’s critic.yaml elevates the reviewer into pure architecture. A Drafter produces the work, a separate Critic scores it, and the orchestrator loops the interaction until the output earns strict confidence.
- name: "CriticAgent"
generateContentConfig:
responseMimeType: "application/json"
outputSchema:
type: "OBJECT"
properties:
message: { type: "STRING" }
confidence: { type: "INTEGER" } # below 85 → the answer loops
required: ["message", "confidence"]The orchestrator operates under an absolute contract: If the confidence score is 85 or above, return the JSON verbatim to the user. If below, cite the Critic’s specific concerns back to the Drafter and repeat the process, capped strictly at three rounds. The moment quality became a parsed JSON field, it became executable architecture.
Two fundamental design choices support this structure. First, the Critic must operate as a separate agent. The cognitive blind spots that generate an error will naturally approve it; the maker must never grade the making. Second, the Critic’s YAML configuration reflects a hard architectural constraint discovered through testing: the ADK deadlocks any single agent holding both an outputSchema and transfer powers. Consequently, the schema resides on a leaf agent entirely stripped of transfer capabilities.
Observe the complete loop executing:
We maintain strict transparency regarding this gate. We provoked the live syndicate with three distinct tests: a plain question, a false premise, and an impossible request. In every instance, the Drafter cleared the 85-point threshold in a single round, autonomously correcting the premise and proving the impossibility. (The unedited run is included in the download.) When utilizing strong models, the loop functions as vital insurance rather than a daily crutch. The gate generally remains green, yet you maintain the architecture because the day it flashes red is the day it proves its absolute necessity.
Equip yourself with the materials. Download the verbatim syndicate, complete with the live run and detailed design notes: critic-workflow.md. Retrieve the skeptical QA persona: qa-persona-prompt.md. The five-dials quick reference card is included within module 01’s gide-checklist.md. Execute the system live via npm run syndicate:critic -- "your question". This utilizes the exact setup from module 01 and requires no database.
Test manually to build intuition, then construct the automated tester. The five variables train your diagnostic capability; the adversarial persona industrializes your systemic suspicion; the critic loop executes both while you sleep. However, none of these mechanisms possess the capacity to remember last month’s patient, last week’s lab values, or yesterday’s definitive decision. That requires an engineered memory architecture. Module 04 constructs exactly that.