Module 2 · Tools, Knowledge, Memory & Research Design · scripted
Exercise: The Design Studio
Turn the method on your own design
The case studies just showed you the method: one goal, several languages, watch what each language can and cannot say. Now you apply it to an agent you want — with an analyst on the other side of the table. You describe the agent, write a first tool set in the familiar plain-text format, and a generator takes your language of action apart: what it can say, what it cannot, where the parameters fail to chain — and then proposes rival languages across the whole design spectrum, each with the same honest analysis.
The studio loop
Where to run it
Open the Design Studio →. Two boxes:
the agent (what it should accomplish, for whom, acting on what world)
and the tool set (same tool: / description: / parameters: format
as before; the template menu has starting points). One button generates
the full analysis as a structured report — tables, chains, rival
designs — and a follow-up box underneath keeps the conversation going:
every question you ask gets its own analysis appended below. In an LLM
of your choice, the same exercise works by asking for exactly those
sections — the analysis of your design, three alternatives along the
open-to-bounded spectrum, and can-say / can’t-say / safety
properties for each.
What comes back
Step 1 · Your agent, first draft, full analysis
Describe an agent from your research domain and draft its tool set — four to seven tools, honest first attempt, don't polish. Generate the analysis and read all of it slowly. Pay particular attention to two places: the gives back column (did you ever say what your tools return? could the chains actually connect?) and the cannot say table (which of those missing tasks do you actually want?).
Capture — end of Step 1
Step 2 · Interrogate the spectrum
Now use the follow-up box to push. Ask at least three questions that the report didn't already answer. Good pushes: combine A's naming with C's decomposition · what if every tool returned ids instead of prose? · make the most dangerous tool safe without deleting it · what does the one-tool open-ended version look like, and what would we lose? Each answer arrives as a fresh analysis — treat it as a design partner you are steering, not an oracle you are accepting.
Capture — end of Step 2
Step 3 · Commit, in the language of the dimensions
Choose your final design — original, an alternative, or a hybrid you negotiated through follow-ups. Then justify the choice in one short paragraph that uses the dimensions explicitly: where it sits on the open↔bounded spectrum and why that position fits your task; which naming decisions carry the most meaning; how results chain into the next call's parameters; and which tools would need a human gate, in the sense you felt in the Forge.
Capture — end of Step 3
Deliverable
Three captures. In the comparison we will read the cannot say rows aloud — they are the sharpest evidence of what tool design actually decides — then look at the follow-up questions that most changed a design, and the final spectrum positions people chose for their own research agents, with their reasons.
- The analyst proposed a reframing (alternative C) — a different set of nouns and verbs for the same goal. Did anyone's reframing beat their original? What made it better: the verbs, the nouns, or the decomposition?
- Where did "blast radius" and "expressiveness" pull in opposite directions, and how did you split the difference?
- What did writing the GIVES BACK column teach you that writing the parameters never did?
This is how it was built
The Design Studio feels like a purpose-built analysis engine — structured reports, fixed tables, a spectrum, an interrogation loop. Here is the whole machine: two text boxes, one system prompt, one growing conversation, and a markdown renderer. There is no analysis code. No parser reads your tool set; no algorithm computes the spectrum. Every capability you just used was purchased with prompt design, and it is worth seeing exactly how.
The architecture
Look at where the arrows loop: the follow-up box feeds back into the same conversation. The studio is not a pipeline of separate calls — it is one conversation that grows, exactly like every conversation in this course, wearing a document as a costume.
The system prompt is the application
Here it is, verbatim — the entire analytical engine. (The
{tool format} splice is the same format specification string the
Tool Playground uses; the prompt is assembled from shared parts, the
way code is assembled from libraries.)
The studio system prompt · part 1 — persona and contract
You are a tool-language design analyst for AI agents. The user gives you (1) a description of the agent they want and (2) a tool set in this plain-text format:
Your job is to analyze the tool set AS A LANGUAGE OF ACTION — the words this agent can "speak" to the world — and to propose alternative languages for the same agent. Respond in clean, well-structured MARKDOWN (headings, tables, short code blocks). No preamble, no closing pleasantries — the report IS the response.
Three moves already: a persona (an analyst, not an assistant — you built this pattern in the pattern library), the spliced format spec (so it can read what you write without any parser), and a line that kills the chatbot register — the report IS the response. Without that line you get "Great question! I'd be happy to analyze…" wrapped around everything.
The studio system prompt · part 2 — the skeleton
Produce exactly these sections:
## The design as written One sentence on the agent's purpose as you understand it. Then a table: | tool | what it says | takes | gives back |. "Gives back" is your inference of what the result must contain for the language to work — flag any tool whose result shape is unclear.
## What this language can say 3–4 representative tasks this agent could complete, each as a compact chain showing parameter flow, e.g.: `list_participants → (ids) → schedule_session(participant=…) → update_stage(participant=…, stage=scheduled)` One line under each chain: where a result feeds the next call's parameters — and where the model must invent or remember a value because nothing returned it.
## What it cannot say 3–4 DESIRABLE tasks (given the agent description) that this tool set cannot complete. For each: the task, and the exact missing word — a verb that doesn't exist, a parameter that can't be filled, or a result that never comes back. Present as a table: | desirable task | why the language can't say it | the missing word |
## Friction in the language Short bullets: naming problems (misleading, inconsistent, too vague or too narrow), parameter-chaining gaps (tool A returns X but tool B needs Y), granularity problems (one tool doing too much or too little).
This is a template pattern doing the heavy lifting. The headings are dictated word for word, so every report has the same skeleton — which is what makes reports comparable across runs and across classmates, and what let this lesson promise you, in advance, exactly what would come back. Notice also how much analysis is smuggled into the section definitions themselves: "gives back" is defined in the prompt ("your inference of what the result must contain"), the chain notation is demonstrated with a worked example rather than described, and "the missing word" teaches the model the course's own vocabulary so the report speaks it back to you.
The studio system prompt · part 3 — the alternatives
## Alternative languages Exactly three alternative designs, each with a short evocative name:
### Alternative A — more open-ended ### Alternative B — more bounded ### Alternative C — a different framing entirely (For C: reconceive the problem — different nouns, different verbs, a different decomposition, not just looser/tighter versions.)
## The spectrum A comparison table of the original + all three alternatives: | design | expressiveness | predictability | blast radius | | when to choose it | Keep cells to a few words.
## Questions back to the designer Two or three pointed questions whose answers would change which design wins.
The constraints here are all load-bearing. "Exactly three" prevents the model from writing one lazy variant or ten shallow ones. The parenthetical on C — "not just looser/tighter versions" — exists because without it, alternative C collapses into a fourth point on the open↔bounded line instead of a genuine reframing; that one sentence is the difference between a spectrum and a triangle. "1–2 honest losses" forces the analysis to admit trade-offs — models will otherwise sell every alternative as strictly better. And the safety-properties recipe (worst plausible action / impossible by construction / which tool to gate) is the Forge's approval checkbox turned into an analytical obligation.
The follow-up loop is one sentence
How does "what about…?" keep working, round after round, without any special handling? The system prompt ends with one paragraph:
The studio system prompt · part 4 — staying in character
For follow-up messages from the user: stay in this analytical mode and answer in the same structured markdown style. If they propose or ask for a new design variation, give it the full alternative treatment (tool set in a code block + can say / can't say / safety properties), and update the spectrum table if it changes.
And the code beneath it is nothing but the trajectory you already know: every round, the entire conversation — system prompt, your original brief and tools, every report, every question — is sent again, and the new report is appended. One big prompt, growing.
The conversation after two follow-ups
The interface renders assistant turns as documents and user turns as small question chips — that is the entire difference between "a chat" and "a structured analysis tool." One practical note completes the picture: reports this size need room, so this one conversation runs with a raised output-token ceiling — a reminder that output budget is a design parameter like any other.
Steal this architecture. Pick a recurring analysis from your own research — reviewing an experiment design, critiquing a survey instrument, stress-testing a study protocol — and write the studio system prompt for it: a persona, a fixed section skeleton with the analysis smuggled into the section definitions, a bounded number of alternatives with mandatory honest losses, and one staying-in- character paragraph for follow-ups. Run it in any LLM. You have just built an application.