Module 2 · Tools, Knowledge, Memory & Research Design · scripted

Exercise: The Design Studio

40 minoutcomes: tool-design

Turn the method on your own design

The case studies just showed you the method: one goal, several languages, watch what each language can and cannot say. Now you apply it to an agent you want — with an analyst on the other side of the table. You describe the agent, write a first tool set in the familiar plain-text format, and a generator takes your language of action apart: what it can say, what it cannot, where the parameters fail to chain — and then proposes rival languages across the whole design spectrum, each with the same honest analysis.

The studio loop

your design→ ANALYZE what it can say (chains of calls) what it CANNOT say (the missing words) friction: naming, chaining, granularity
PROPOSE three rival languages: more open · more bounded · reframed entirely each with can/can't + safety properties
YOU ASK "what about…?" — and it analyzes again

Where to run it

Open the Design Studio →. Two boxes: the agent (what it should accomplish, for whom, acting on what world) and the tool set (same tool: / description: / parameters: format as before; the template menu has starting points). One button generates the full analysis as a structured report — tables, chains, rival designs — and a follow-up box underneath keeps the conversation going: every question you ask gets its own analysis appended below. In an LLM of your choice, the same exercise works by asking for exactly those sections — the analysis of your design, three alternatives along the open-to-bounded spectrum, and can-say / can’t-say / safety properties for each.

What comes back

## The design as writtenthe tools as a table — including what each must GIVE BACK to work
## What this language can saychains: call → (result) → next call
## What it cannot saydesirable tasks + the missing word
## Friction in the languagenaming · chaining · granularity
## Alternative languagesA more open · B more bounded · C reframed — each with tools, can/can't, safety properties
## The spectrumexpressiveness · predictability · blast radius · when to choose it
## Questions backwhat the analyst wants YOU to decide

Step 1 · Your agent, first draft, full analysis

Describe an agent from your research domain and draft its tool set — four to seven tools, honest first attempt, don't polish. Generate the analysis and read all of it slowly. Pay particular attention to two places: the gives back column (did you ever say what your tools return? could the chains actually connect?) and the cannot say table (which of those missing tasks do you actually want?).

Capture — end of Step 1

Your brief and tool set, verbatim.
One chain from "can say" where a result feeds the next call —
and one place the model would have to INVENT a value because
nothing returned it.
The "cannot say" row that stung: a task you genuinely want that
your language cannot express.

Step 2 · Interrogate the spectrum

Now use the follow-up box to push. Ask at least three questions that the report didn't already answer. Good pushes: combine A's naming with C's decomposition · what if every tool returned ids instead of prose? · make the most dangerous tool safe without deleting it · what does the one-tool open-ended version look like, and what would we lose? Each answer arrives as a fresh analysis — treat it as a design partner you are steering, not an oracle you are accepting.

Capture — end of Step 2

Your best follow-up question and the design change it produced.
One claim in the analysis you DISAGREE with, and your argument.

Step 3 · Commit, in the language of the dimensions

Choose your final design — original, an alternative, or a hybrid you negotiated through follow-ups. Then justify the choice in one short paragraph that uses the dimensions explicitly: where it sits on the open↔bounded spectrum and why that position fits your task; which naming decisions carry the most meaning; how results chain into the next call's parameters; and which tools would need a human gate, in the sense you felt in the Forge.

Capture — end of Step 3

The final tool set, verbatim.
The justification paragraph — spectrum position, naming,
chaining, gating.

Deliverable

Three captures. In the comparison we will read the cannot say rows aloud — they are the sharpest evidence of what tool design actually decides — then look at the follow-up questions that most changed a design, and the final spectrum positions people chose for their own research agents, with their reasons.

As you compare
  • The analyst proposed a reframing (alternative C) — a different set of nouns and verbs for the same goal. Did anyone's reframing beat their original? What made it better: the verbs, the nouns, or the decomposition?
  • Where did "blast radius" and "expressiveness" pull in opposite directions, and how did you split the difference?
  • What did writing the GIVES BACK column teach you that writing the parameters never did?

This is how it was built

The Design Studio feels like a purpose-built analysis engine — structured reports, fixed tables, a spectrum, an interrogation loop. Here is the whole machine: two text boxes, one system prompt, one growing conversation, and a markdown renderer. There is no analysis code. No parser reads your tool set; no algorithm computes the spectrum. Every capability you just used was purchased with prompt design, and it is worth seeing exactly how.

The architecture

Look at where the arrows loop: the follow-up box feeds back into the same conversation. The studio is not a pipeline of separate calls — it is one conversation that grows, exactly like every conversation in this course, wearing a document as a costume.

The system prompt is the application

Here it is, verbatim — the entire analytical engine. (The {tool format} splice is the same format specification string the Tool Playground uses; the prompt is assembled from shared parts, the way code is assembled from libraries.)

The studio system prompt · part 1 — persona and contract

You are a tool-language design analyst for AI agents. The user gives you (1) a description of the agent they want and (2) a tool set in this plain-text format:

{tool format — the same tool:/description:/parameters: spec
you have been writing all along, spliced in verbatim}

Your job is to analyze the tool set AS A LANGUAGE OF ACTION — the words this agent can "speak" to the world — and to propose alternative languages for the same agent. Respond in clean, well-structured MARKDOWN (headings, tables, short code blocks). No preamble, no closing pleasantries — the report IS the response.

Three moves already: a persona (an analyst, not an assistant — you built this pattern in the pattern library), the spliced format spec (so it can read what you write without any parser), and a line that kills the chatbot register — the report IS the response. Without that line you get "Great question! I'd be happy to analyze…" wrapped around everything.

The studio system prompt · part 2 — the skeleton

Produce exactly these sections:

## The design as written One sentence on the agent's purpose as you understand it. Then a table: | tool | what it says | takes | gives back |. "Gives back" is your inference of what the result must contain for the language to work — flag any tool whose result shape is unclear.

## What this language can say 3–4 representative tasks this agent could complete, each as a compact chain showing parameter flow, e.g.: `list_participants → (ids) → schedule_session(participant=…) → update_stage(participant=…, stage=scheduled)` One line under each chain: where a result feeds the next call's parameters — and where the model must invent or remember a value because nothing returned it.

## What it cannot say 3–4 DESIRABLE tasks (given the agent description) that this tool set cannot complete. For each: the task, and the exact missing word — a verb that doesn't exist, a parameter that can't be filled, or a result that never comes back. Present as a table: | desirable task | why the language can't say it | the missing word |

## Friction in the language Short bullets: naming problems (misleading, inconsistent, too vague or too narrow), parameter-chaining gaps (tool A returns X but tool B needs Y), granularity problems (one tool doing too much or too little).

This is a template pattern doing the heavy lifting. The headings are dictated word for word, so every report has the same skeleton — which is what makes reports comparable across runs and across classmates, and what let this lesson promise you, in advance, exactly what would come back. Notice also how much analysis is smuggled into the section definitions themselves: "gives back" is defined in the prompt ("your inference of what the result must contain"), the chain notation is demonstrated with a worked example rather than described, and "the missing word" teaches the model the course's own vocabulary so the report speaks it back to you.

The studio system prompt · part 3 — the alternatives

## Alternative languages Exactly three alternative designs, each with a short evocative name:

### Alternative A — more open-ended ### Alternative B — more bounded ### Alternative C — a different framing entirely (For C: reconceive the problem — different nouns, different verbs, a different decomposition, not just looser/tighter versions.)

For EACH alternative give:
The full tool set in the same plain-text format, in a code block.
"Can now say": 2–3 chains newly possible.
"Can no longer say / still can't say": 1–2 honest losses or gaps.
"Safety properties": what a confused or misdirected agent could
do with this language (worst plausible action), what the design
makes IMPOSSIBLE by construction, and which single tool you
would gate behind human approval and why.

## The spectrum A comparison table of the original + all three alternatives: | design | expressiveness | predictability | blast radius | | when to choose it | Keep cells to a few words.

## Questions back to the designer Two or three pointed questions whose answers would change which design wins.

The constraints here are all load-bearing. "Exactly three" prevents the model from writing one lazy variant or ten shallow ones. The parenthetical on C — "not just looser/tighter versions" — exists because without it, alternative C collapses into a fourth point on the open↔bounded line instead of a genuine reframing; that one sentence is the difference between a spectrum and a triangle. "1–2 honest losses" forces the analysis to admit trade-offs — models will otherwise sell every alternative as strictly better. And the safety-properties recipe (worst plausible action / impossible by construction / which tool to gate) is the Forge's approval checkbox turned into an analytical obligation.

The follow-up loop is one sentence

How does "what about…?" keep working, round after round, without any special handling? The system prompt ends with one paragraph:

The studio system prompt · part 4 — staying in character

For follow-up messages from the user: stay in this analytical mode and answer in the same structured markdown style. If they propose or ask for a new design variation, give it the full alternative treatment (tool set in a code block + can say / can't say / safety properties), and update the spectrum table if it changes.

And the code beneath it is nothing but the trajectory you already know: every round, the entire conversation — system prompt, your original brief and tools, every report, every question — is sent again, and the new report is appended. One big prompt, growing.

The conversation after two follow-ups

system:[the studio system prompt]
user:THE AGENT I WANT: … THE TOOL SET: … Analyze this language of action and propose the alternatives.
assistant:[report 1 — full skeleton]
user:What if every tool returned ids instead of prose?
assistant:[report 2 — same register, updated spectrum]
user:Combine A's naming with C's decomposition.
assistant:[report 3 — the hybrid, full alternative treatment]

The interface renders assistant turns as documents and user turns as small question chips — that is the entire difference between "a chat" and "a structured analysis tool." One practical note completes the picture: reports this size need room, so this one conversation runs with a raised output-token ceiling — a reminder that output budget is a design parameter like any other.

Your turn

Steal this architecture. Pick a recurring analysis from your own research — reviewing an experiment design, critiquing a survey instrument, stress-testing a study protocol — and write the studio system prompt for it: a persona, a fixed section skeleton with the analysis smuggled into the section definitions, a bounded number of alternatives with mandatory honest losses, and one staying-in- character paragraph for follow-ups. Run it in any LLM. You have just built an application.