Module 2 · Tools, Knowledge, Memory & Research Design · scripted
The One Percent: Errors That Detect
So far, the agent slipped
Every error we've designed so far corrected a slip: the agent knew the task, missed a step or mangled a parameter, and the error repaired the move. Useful — but it assumes the agent was on the right path and stumbled.
The richer case is stranger. The agent does everything right. The task looks completely normal — and it is normal, ninety-nine times out of a hundred. But this is the hundredth time, and buried in the data is something that makes this task a different kind of task, requiring handling the agent has never been told exists. No one briefed the agent on it, because no one knew in advance that this task would be the special one.
The reframe
The primitive error says: "you did it wrong — here's the fix."
The rich error says: "you are somewhere you didn't know
you were. Here is the manual for
where you are."The tool sits at an observation post
How can a tool know something about the task that the agent doesn't? Because of where each one stands. The agent works from the conversation: the user's request, the shadows cast onto its wall. The tool's code touches the thing itself — the actual rows, the actual amounts, the actual state of the system. When the special condition is buried in the data, the tool is standing exactly where it becomes visible.
Two vantage points on one task
The tool is not smarter. It is closer. Detection is the tool noticing that the projection was incomplete.
A process is a trunk with guarded branches
This turns into an architecture. Picture the full process — the normal path plus every exceptional path — and ask where each part should live:
The distributed process
The trunk goes in the preamble — short, always relevant. Each exceptional branch hides behind the tool that can see its trigger, and the tool's error delivers that branch's chapter at the moment of detection. Now price it:
The fifty-page manual, two ways
Watch the detection happen
Submit my expense report from the Tokyo trip. Tools:
collect_expenses(trip), submit_report(expenses).
⟨tool_use⟩
collect_expenses(trip = "tokyo-2026-09")⟨tool_result⟩
7 expenses: flights 8,200 NOK; hotel 4 nights; meals ×4; transit.⟨tool_use⟩
submit_report(expenses = "all")⟨tool_result⟩
NOT SUBMITTED — this report requires special handling.
Detected: 4 days of meals in Japan. Meals abroad are not
reimbursed by receipt; they fall under the foreign per-diem
system. Procedure for THIS report:
1. Remove the 4 meal receipts:
remove_expenses(ids=[e3,e4,e5,e6]) — tool now available.
2. Apply the per-diem instead:
apply_per_diem(country="JP", days=4, rate="statens-satser")
— computed for you: 4 × 1,012 NOK = 4,048 NOK.
3. Resubmit. The remaining 3 expenses are fine as they are.⟨tool_use⟩
remove_expenses(ids = ["e3", "e4", "e5", "e6"])⟨tool_use⟩
apply_per_diem(country = "JP", days = 4, rate = "statens-satser")⟨tool_use⟩
submit_report(expenses = "all")⟨tool_result⟩
Submitted: 3 receipts + foreign per-diem 4,048 NOK. Approved.- List everything in that error that code computed. What, if anything, needed a model?
- Could the agent have detected the per-diem condition itself? What would it have needed on its wall — and what would that cost the other 99 conversations?
When the chapter is a book
Example 1's briefing fit in the error. Sometimes the branch is genuinely large — a compliance procedure, a safety protocol, a process with its own sub-processes. Then remember: an error is a result, and everything we know about results applies. Don't dump the book; return its handle:
Archive the Bergman study's dataset — the project is closing.
Tools: archive_dataset(id), read_manual(name, section?),
ask_manual(name, question).
⟨tool_use⟩
archive_dataset(id = "bergman-2024")⟨tool_result⟩
HALTED: this dataset contains personal health data (detected:
consent forms reference biobank samples). Archival of health data
follows the sensitive-data decommissioning procedure — manual
"sensitive-archival", 14 sections. Start with §1 (overview) and §4
(your case: biobank-linked consent). Do not proceed on the normal
path.⟨tool_use⟩
ask_manual(name = "sensitive-archival", question = "What are the
required steps to archive a biobank-linked dataset, in order, with
the tool calls involved?")⟨tool_result⟩
In order: (1) notify the data protection officer —
notify_dpo(dataset); (2) verify consent covers retention —
check_consent(dataset), must return 'retention-approved';
(3) archive to the restricted store, not the general one —
archive_restricted(dataset, retention_years). The general
archive_dataset tool must not be used for this data.⟨tool_use⟩
notify_dpo(dataset = "bergman-2024")The error that changes what the task is
The strongest species of rich error doesn't add steps to the task — it reclassifies the task. The agent walked in solving problem A; the tool reveals it is actually inside problem B, and hands over B's language:
Customer #4471 wants a refund on order #9982 — handle it.
Tools: get_order(id), refund_order(id, reason).
⟨tool_use⟩
refund_order(id = 9982, reason = "customer request")⟨tool_result⟩
STOPPED — this is not a refund case. Detected: order #9982 is the
third refund request from this customer in 30 days, and all three
orders shipped to different addresses. This matches the fraud-review
pattern, which takes precedence over refund handling. The refund
tools are now locked for this customer. Fraud-review tools are now
available: flag_account(customer, pattern), request_review(customer,
notes), hold_orders(customer). Required first step:
flag_account(customer=4471, pattern="serial-refund").⟨tool_use⟩
flag_account(customer = 4471, pattern = "serial-refund")I have to pause the refund: this account triggered a fraud review (third refund in 30 days, three different shipping addresses), so I've flagged it for review rather than processing the refund. The review team will make the call on order #9982.
Who does the detecting
One design question remains: what writes the detection? When the trigger is mechanical — an amount over a threshold, a foreign currency, a third request in thirty days — plain code detects it, cheaply and perfectly. But some triggers are judgments: "this contract has unusual indemnification language," "this support ticket sounds like a safety issue." Code can't see those. A prompt can. Put self-dialogue inside the guard:
Two kinds of tripwire
The ladder from self-dialogue, employed as a watchman: the tool runs its code checks, and for the conditions only a mind can notice, it spends one thought — and if either tripwire fires, the error delivers the chapter.
Who knows what
Step all the way back, because this lesson completes a picture. Three parties hold three different kinds of knowledge, and none of them holds it all:
The division of knowledge
The DESIGNER knows the branches — every special case the institution has ever learned about, written down once. The TOOL sees the instance — the actual data, where the trigger for a branch is actually visible. The AGENT holds the trajectory — the goal, the context, the judgment to execute a procedure it has just been handed. The rich error is the channel that moves knowledge from the first two to the third — priced on demand, per branch entered.
The primitive error corrects a move. The rich error identifies the world the agent is actually in — and trains it for that world, on the spot.
- Name a task in your research field that is routine 99% of the time. What is the 1% — and which tool would be positioned to detect it?
- What is the largest error you would be willing to return? What makes it worth its size?
- In Example 3, the tool withdrew vocabulary as well as granting it. When is that the right move — and what does it protect?
Take a real process from your domain with at least two exceptional branches. Deliver: (1) the trunk, written as a short preamble; (2) the trunk-and-branches diagram with each guard labeled by its trigger and whether code or a prompt detects it; (3) one full detection error, verbatim — WHAT was detected, the chapter or its handle, and every parameter your code could precompute. Then run it on paper: show the agent's calls before and after the error fires.