Just-in-Time Training: Results That Instruct

A wall computers have, and LLMs don't

To see the next idea clearly, we need one piece of classical computing — and you don't need to be a computer scientist to get it, because you use the distinction every day. Think of a recipe card and a bag of groceries. The card tells you what to do; the groceries are what you do it to. You would never confuse them — no one reads a label on a flour bag that says "now pour everything on the floor" and obeys it, because groceries don't get to give orders.

Classical computers are built on exactly this separation: instructions (the program, which is executed) and data (what the program works on, which is only ever read). The machinery keeps them apart, and enormous security consequences hang on that wall. Nearly every classic attack on computer systems is an attempt to break it — to smuggle something executable in through the door marked data, so that what should have been inert cargo seizes control.

In a conversation, there is no wall

Now the uncomfortable, powerful fact about LLMs: there is no such separation. There is only the prompt — one stream of words — and the conversation mixes instruction and data together by its very nature. We can try to mark boundaries: dashed lines, headers, "the following is only data:" — but those markings are themselves just more words on the same wall. Formatting is a polite request, not a fence.

In a conversation there is only one stream: words.
Anything that enters it can instruct.

And now recall the key fact from the last lesson: a tool result is a user message. Put the two together and something drops out of the machinery — a tool result can be more than information. A tool result can be an instruction.

This is both an opportunity and a security risk. The risk is real — anything a tool reads from the world, a web page, an email, a document, enters the conversation with the power to instruct — and it deserves its own careful treatment. Hold onto it. Today we take the opportunity.

The opportunity: the tool gets to say "no, go this way"

You have done this yourself in a chat: "No, not like that — go this other direction instead." And the model turned. The user seat steers. Since tool results occupy that same seat, the tool's designer holds the same steering wheel — and can use it at exactly the moment it's needed. Some patterns:

⟨tool_use⟩
edit_file(path = "config.yaml", old = "databse", new = "database")

⟨tool_result⟩
ERROR: You must read a file before editing it. Call
read_file("config.yaml") first, then retry the edit.

⟨tool_use⟩
read_file(path = "config.yaml")

⟨tool_result⟩
[contents: …connection: databse_url: postgres://…]

⟨tool_use⟩
edit_file(path = "config.yaml", old = "databse_url",
          new = "database_url")

⟨tool_use⟩
connect(database = "student-records")

⟨tool_result⟩
Connected. You now have access to two new tools:
query(sql) — read-only SELECT against student-records;
export_csv(query_id) — export a completed query's rows.

⟨tool_use⟩
query(sql = "SELECT count(*) FROM enrollments WHERE cohort =
'2026-spring'")

⟨tool_use⟩
measure(sensor = "S-4")

⟨tool_result⟩
BLOCKED: S-4 was last calibrated 26 hours ago; protocol requires
calibration within 24h of measurement. Run calibrate("S-4") first.
Note: after calibration, discard the first reading as warm-up.

⟨tool_use⟩
calibrate(sensor = "S-4")

⟨tool_result⟩
Calibrated. Reminder: take two measurements and discard the first.

Just-in-time training

Name what all three examples are doing: injecting training on demand into the conversation. A tool result that instructs is training that arrives exactly when it's relevant — not in a system message the agent must carry from turn zero, but at the moment of need. And it comes in strengths:

Now connect this to the economics, because this is where the design power lives. We learned that everything in the preamble — every rule, every description — is reprinted on every iteration, needed or not. Just-in-time training moves rules out of the always-paid preamble and prices them on demand: the rule about calibration costs nothing until the day an agent tries to skip it. Front-load the rules an agent always needs; put behind the tools the rules it needs sometimes — and let the tools teach them at the moment of violation.

One channel, many steering hands

Step back to the unifying picture. The conversation is a trajectory, and we've now counted the hands that can bend it: the user's messages, the model's own outputs (self-conditioning), the alignment baked in before the conversation began — and now, deliberately, the tools. A result is a shadow cast onto the agent's wall, and there is no rule that a shadow may only inform. The tool designer sits in the user's seat every time a result returns, and a well-designed tool uses that seat the way a good mentor does: silent when things go well, specific exactly when they don't.