Exercise: The Memory Lab

The goal: give a chatbot a memory — and own both halves

You know what memory really is now: files between conversations, an extractor that writes them, a recaller that brings them back. In this exercise you run all of it — a live chat whose memory system is built from two prompts that you design: the extraction prompt (what counts as a memory, and how the files are organized) and the recall prompt (the strategy for finding what matters before each reply). The machinery is real: two background agents with filesystem tools, working over a memory store you can open and read at any moment.

How the two agents work

The extractor runs in the background after each reply, receiving the full current conversation. Only one instance ever runs: if the conversation moves on while it is busy, it simply runs again when it finishes, on the newest complete transcript — so it can file several memories at once. It has the full filesystem vocabulary — list, search, read, write, and edit — so it can update and correct the memory, not just append to it.

The recaller runs when you send a message, before the assistant replies. Watch the status bubble: instead of "thinking…", you'll see its actual moves — searching for "thesis"… reading memories/projects.md… — until it commits to a list of recalled memories. Those are inserted under your message, visibly, as the chips the reply was built on.

Where to run it

Open the Memory Lab →. The two prompts sit at the top; the memory files and every extraction run (its tool calls, its reasoning, its summary) are browsable below the chat. In an LLM of your choice, the same experiment works by hand with two conversations: one is the chat, the other is the memory system — you paste the transcript in and ask it what to remember, then ask it what's relevant before each new task, and carry the answers across yourself. The Lab simply automates that loop.

Step 1 · Watch memory happen

Keep the default prompts. Have a genuinely memorable conversation — introduce yourself, your research topic, a preference or two, a decision ("let's call the study STAGE-2 from now on"). Watch the extraction runs appear and open the files they wrote. Then press New conversation — the chat forgets everything; the files survive — and ask something that deserves recall: "what should I cook for the celebration when my study gets approved?" Watch the status bubble search before the reply arrives.

Step 2 · Design the extraction

Now replace the default extraction prompt with your own design. Decide what a memory IS for your use case, and how the store is organized — this is a containers-and-conventions decision, and the searching-recaller is your reader: name files with the words recall queries will use. Then stress it: tell the assistant something, and later correct it ("actually, the study is called STAGE-3 now"). Open the files: did the extractor edit the old fact, or append a contradiction? Does your prompt even tell it which to do?

Step 3 · Design the recall — then hunt its failures

Rewrite the recall prompt with an explicit strategy, and then try to break it both ways. Force a false positive: ask something harmless and see whether irrelevant memories get dragged in anyway — does the reply bend toward them? Force a miss: ask about something you know is stored, phrased in words that don't appear in any file — synonyms are the classic hole. Each failure is a design lesson: the false positive is a relevance-threshold problem in your prompt; the miss is a naming problem in your files.

Deliverable

Three captures. In the comparison we look at extraction prompts side by side (what counted as "worth remembering"?), the correction test results — edit versus append is the whole staleness debate from the relay, replayed inside one afternoon — and the engineered failures, which are this exercise's real product: every commercial "memory" feature has exactly these failure modes, invisibly.