Module 1 · Foundations of Agentic AI · scripted

Exercise: The Window Lab

35 min

The goal: feel the wall's edges

You've now seen both constraints — the window has edges, and every iteration re-bills the whole conversation. In this exercise you control both dials directly and watch what they do to an agent's ability to solve a problem. The instrument is the Window Lab: the Tool Playground with a context window you set.

How the window is enforced

You choose a window size in tokens. Before every prompt is
sent, the Lab walks backward from the newest message, keeping
what fits — and EVICTS the older messages that don't.

One exception: the task message always survives. Everything
else competes for the space that remains.

Open the Window Lab → — everything from the Tool Playground is here (the tool format, templates, you-play-the-computer results), plus three new instruments:

The meters: total input tokens consumed across the run, and the same total with prefix caching — a synthetic 0.1× discount on tokens that match the previous prompt's prefix. Watch the two numbers diverge, and notice when the discount disappears: eviction rewrites the front of the prompt, and a rewritten prefix is a cache miss.
The numbered prompts ("show prompts"): every prompt actually sent, in the script format, with its token count, its cached cost — and a ⚠ marker counting what was evicted.
Synthetic results: beside the result box, ✨ buttons ask a second LLM to write the tool result for you — short, medium, long, or overflow (deliberately larger than the space that remains). It sees the whole conversation and generates something realistic that keeps the task achievable. Use these to stress the window without typing forty lines yourself.

Step 1 · Baseline

Load a template (Filesystem explorer or Search & read), set the window generously — 8,000 or more — and run the task to completion, playing the computer with medium results. Record the meters.

Capture — end of Step 1

Window size, number of prompts, total tokens, cached total.

From the prompt log: how much of prompt 5's cost was prefix —

and what would this run cost without caching?

Step 2 · Shrink until it breaks

Same task. Halve the window and rerun. Halve it again. Somewhere on the way down, the agent stops being able to solve the problem — find that point, and more importantly, watch how it fails. Eviction failures have signatures: the agent re-reads a file it already read (the result was evicted), contradicts an earlier finding, loops on a step it already did, or forgets what it concluded and starts over. Use ✨ long and overflow results to force evictions on demand.

Capture — end of Step 2

The largest window at which the run FAILED, and the smallest

at which it succeeded.

The failure signature: which evicted message did the agent

need, and what did it do without it? (The ⚠ markers in the

prompt log show you exactly what was lost, and when.)

What happened to the caching discount as evictions began?

Step 3 · Redesign to survive the small window

Now the real design challenge: go back to the window size that broke the agent — and make the task solvable there anyway, by redesigning what enters the conversation. Everything you control is on the table:

The levers

the TOOL DESCRIPTIONS — tighter, fewer tokens, still clear

the TASK — restated compactly; is every sentence earning

its place in the one message that always survives?

the RESULTS — you play the computer, so play a well-designed

one: return summaries instead of dumps, counts instead of

listings, a handle ("42 files — ask for a folder") instead

of contents

the TOOLS THEMSELVES — would a different tool need smaller

results? (a search tool instead of read-everything?)

Iterate until the same task completes inside the window that previously broke it.

Capture — end of Step 3

Before/after: the failing design and the surviving one.

The single change that bought the most room — and its cost

in tokens saved per prompt.

Final meters for the surviving run, with and without caching.

Deliverable

Three captures. We will compare together: the failure signatures people observed, the window sizes where their tasks broke, the redesigns that survived — and the meters, because the cheapest run in the room and the most expensive will differ by more than you expect.

As you compare

The task message always survives eviction here. What ELSE, in your runs, deserved that protection — and what does that suggest about designing what goes where?
When evictions began, caching died with them. Say precisely why — and what that implies about how real systems should manage a full window.