Module 2 · Tools, Knowledge, Memory & Research Design · scripted

The One Percent: Errors That Detect

30 minoutcomes: tool-design, agent-architectures

So far, the agent slipped

Every error we've designed so far corrected a slip: the agent knew the task, missed a step or mangled a parameter, and the error repaired the move. Useful — but it assumes the agent was on the right path and stumbled.

The richer case is stranger. The agent does everything right. The task looks completely normal — and it is normal, ninety-nine times out of a hundred. But this is the hundredth time, and buried in the data is something that makes this task a different kind of task, requiring handling the agent has never been told exists. No one briefed the agent on it, because no one knew in advance that this task would be the special one.

The reframe

The primitive error says:  "you did it wrong — here's the fix."

The rich error says:       "you are somewhere you didn't know
                            you were. Here is the manual for
                            where you are."

The tool sits at an observation post

How can a tool know something about the task that the agent doesn't? Because of where each one stands. The agent works from the conversation: the user's request, the shadows cast onto its wall. The tool's code touches the thing itself — the actual rows, the actual amounts, the actual state of the system. When the special condition is buried in the data, the tool is standing exactly where it becomes visible.

Two vantage points on one task

THE AGENT sees:"Submit my expense report from the Tokyo trip." A projection. Nothing about it says "special."
THE TOOL sees:line 7: per-diem, JPY, 4 days, foreign tax treaty applies — the one detail that reclassifies everything.

The tool is not smarter. It is closer. Detection is the tool noticing that the projection was incomplete.

A process is a trunk with guarded branches

This turns into an architecture. Picture the full process — the normal path plus every exceptional path — and ask where each part should live:

The distributed process

The trunk goes in the preamble — short, always relevant. Each exceptional branch hides behind the tool that can see its trigger, and the tool's error delivers that branch's chapter at the moment of detection. Now price it:

The fifty-page manual, two ways

FRONT-LOADEDGUARDED
The full process — trunk andThe trunk: half a page,
every branch — rides everyalways present.
iteration of every
conversation. Fifty pages,The 99% pay nothing more —
paid always, needed rarely.ever. The 1% pay for exactly the chapter their task entered,
Worse: the agent must judgeat the moment it entered it —
which chapter applies — fromand the JUDGMENT of which
the projection, where thechapter is made by code, from
trigger may not even bethe data, where the trigger
visible.is actually visible.

Watch the detection happen

Example 1 · A normal task, until the tool looks
user

Submit my expense report from the Tokyo trip. Tools: collect_expenses(trip), submit_report(expenses).

assistant
⟨tool_use⟩
collect_expenses(trip = "tokyo-2026-09")
user
⟨tool_result⟩
7 expenses: flights 8,200 NOK; hotel 4 nights; meals ×4; transit.
assistant
⟨tool_use⟩
submit_report(expenses = "all")
user
⟨tool_result⟩
NOT SUBMITTED — this report requires special handling.
Detected: 4 days of meals in Japan. Meals abroad are not
reimbursed by receipt; they fall under the foreign per-diem
system. Procedure for THIS report:
1. Remove the 4 meal receipts:
   remove_expenses(ids=[e3,e4,e5,e6]) — tool now available.
2. Apply the per-diem instead:
   apply_per_diem(country="JP", days=4, rate="statens-satser")
   — computed for you: 4 × 1,012 NOK = 4,048 NOK.
3. Resubmit. The remaining 3 expenses are fine as they are.
assistant
⟨tool_use⟩
remove_expenses(ids = ["e3", "e4", "e5", "e6"])
assistant
⟨tool_use⟩
apply_per_diem(country = "JP", days = 4, rate = "statens-satser")
assistant
⟨tool_use⟩
submit_report(expenses = "all")
user
⟨tool_result⟩
Submitted: 3 receipts + foreign per-diem 4,048 NOK. Approved.
NoteThe agent made no mistake — reread its first two calls; they're exactly what the trunk prescribes. The tool detected which task this actually was, and its "error" was a briefing: the rule, the procedure, and every parameter precomputed for this instance — country, days, rate, even the arithmetic. The agent supplied what remained: execution.
The moment of detection
  • List everything in that error that code computed. What, if anything, needed a model?
  • Could the agent have detected the per-diem condition itself? What would it have needed on its wall — and what would that cost the other 99 conversations?

When the chapter is a book

Example 1's briefing fit in the error. Sometimes the branch is genuinely large — a compliance procedure, a safety protocol, a process with its own sub-processes. Then remember: an error is a result, and everything we know about results applies. Don't dump the book; return its handle:

Example 2 · The error that points into the library
user

Archive the Bergman study's dataset — the project is closing. Tools: archive_dataset(id), read_manual(name, section?), ask_manual(name, question).

assistant
⟨tool_use⟩
archive_dataset(id = "bergman-2024")
user
⟨tool_result⟩
HALTED: this dataset contains personal health data (detected:
consent forms reference biobank samples). Archival of health data
follows the sensitive-data decommissioning procedure — manual
"sensitive-archival", 14 sections. Start with §1 (overview) and §4
(your case: biobank-linked consent). Do not proceed on the normal
path.
assistant
⟨tool_use⟩
ask_manual(name = "sensitive-archival", question = "What are the
required steps to archive a biobank-linked dataset, in order, with
the tool calls involved?")
user
⟨tool_result⟩
In order: (1) notify the data protection officer —
notify_dpo(dataset); (2) verify consent covers retention —
check_consent(dataset), must return 'retention-approved';
(3) archive to the restricted store, not the general one —
archive_restricted(dataset, retention_years). The general
archive_dataset tool must not be used for this data.
assistant
⟨tool_use⟩
notify_dpo(dataset = "bergman-2024")
NoteThe error delivered a pointer into the library — title, size, and which sections match this case — and the agent did what it already knows how to do with any large document: ask it. Detection came from the tool; the curriculum came from the library; the reading strategy came from the agent. Three parties, each doing the part it is positioned for.

The error that changes what the task is

The strongest species of rich error doesn't add steps to the task — it reclassifies the task. The agent walked in solving problem A; the tool reveals it is actually inside problem B, and hands over B's language:

Example 3 · Reclassification
user

Customer #4471 wants a refund on order #9982 — handle it. Tools: get_order(id), refund_order(id, reason).

assistant
⟨tool_use⟩
refund_order(id = 9982, reason = "customer request")
user
⟨tool_result⟩
STOPPED — this is not a refund case. Detected: order #9982 is the
third refund request from this customer in 30 days, and all three
orders shipped to different addresses. This matches the fraud-review
pattern, which takes precedence over refund handling. The refund
tools are now locked for this customer. Fraud-review tools are now
available: flag_account(customer, pattern), request_review(customer,
notes), hold_orders(customer). Required first step:
flag_account(customer=4471, pattern="serial-refund").
assistant
⟨tool_use⟩
flag_account(customer = 4471, pattern = "serial-refund")
assistant

I have to pause the refund: this account triggered a fraud review (third refund in 30 days, three different shipping addresses), so I've flagged it for review rather than processing the refund. The review team will make the call on order #9982.

NoteWatch what crossed the wall: not a correction, not a procedure — a new identity for the task, with a new vocabulary attached and the old vocabulary withdrawn. The framing changed mid-trajectory, and the language changed with it. The agent that started this conversation was doing customer service; the one finishing it is doing fraud triage — and the tool made that call, because the tool could see the pattern.

Who does the detecting

One design question remains: what writes the detection? When the trigger is mechanical — an amount over a threshold, a foreign currency, a third request in thirty days — plain code detects it, cheaply and perfectly. But some triggers are judgments: "this contract has unusual indemnification language," "this support ticket sounds like a safety issue." Code can't see those. A prompt can. Put self-dialogue inside the guard:

Two kinds of tripwire

CODE detectsamount > limit currency ≠ NOK count(refunds, 30d) ≥ 3 — exact, free, instant
A PROMPT detects"Does this contract deviate from our standard terms in any way a lawyer should see?" — a disposable exchange inside the tool, one thought, spent only on judgment code cannot make

The ladder from self-dialogue, employed as a watchman: the tool runs its code checks, and for the conditions only a mind can notice, it spends one thought — and if either tripwire fires, the error delivers the chapter.

Who knows what

Step all the way back, because this lesson completes a picture. Three parties hold three different kinds of knowledge, and none of them holds it all:

The division of knowledge

The DESIGNER knows the branches — every special case the
  institution has ever learned about, written down once.

The TOOL sees the instance — the actual data, where the
  trigger for a branch is actually visible.

The AGENT holds the trajectory — the goal, the context,
  the judgment to execute a procedure it has just been handed.

The rich error is the channel that moves knowledge from the
first two to the third — priced on demand, per branch entered.

The primitive error corrects a move. The rich error identifies the world the agent is actually in — and trains it for that world, on the spot.

The one percent in your domain
  • Name a task in your research field that is routine 99% of the time. What is the 1% — and which tool would be positioned to detect it?
  • What is the largest error you would be willing to return? What makes it worth its size?
  • In Example 3, the tool withdrew vocabulary as well as granting it. When is that the right move — and what does it protect?
Your turn

Take a real process from your domain with at least two exceptional branches. Deliver: (1) the trunk, written as a short preamble; (2) the trunk-and-branches diagram with each guard labeled by its trigger and whether code or a prompt detects it; (3) one full detection error, verbatim — WHAT was detected, the chapter or its handle, and every parameter your code could precompute. Then run it on paper: show the agent's calls before and after the error fires.