Instructing AI by Example · scripted
Walkthrough: Train by Example
Walkthrough: train by example
This is an interactive exercise for a shared screen. The lecture was about teaching by example. This walkthrough is about training by example, and the word is chosen on purpose. Before large language models, if you wanted a computer to learn a task from examples, you trained a custom machine learning model. Today we are going to do the same thing with a pattern of examples and nothing else.
The discipline is strict, and it is the whole point.
The rule
The Pattern Trainer at /exercises/pattern-trainer is built for this: the one permitted instruction is already in place, and the box below it takes only the pattern. In a chat tool, send the one instruction as the first message and the pattern as the second.
- Teach a classifier with labeled examples alone.
- Find the fewest examples that still work.
- Teach a boundary with one more example.
- Teach a procedure, with and without a line of thinking.
- Teach a language nobody has ever spoken.
Step 1 · A classifier from examples alone
Ask the class for a category system with three to five labels and something to sort into them: messages to a small business inbox, items in a lost and found, songs by mood. Write four to six examples, then one left incomplete.
The shape
Here is the shape filled in. The labels are for messages sent to a bike repair shop, and the LLM has to sort a new message.
Complete the pattern. Output only the completion, with no explanation.
Understood.
thing: my back brake squeals every time I stop label: repair
thing: do you have any used kids’ bikes under $150 label: sale
thing: what time do you close on Saturdays label: question
thing: the chain keeps slipping when I shift label: repair
thing: I want to trade in my old bike toward a new one label: sale
thing: my front tire has gone flat twice this week label:
repair
The LLM produced the label and only the label. Nobody told it the three labels existed, and nobody defined them. Five examples did that. If your LLM produces a sentence instead of a label, the pattern is not tight enough; make the examples more uniform and try again.
Step 2 · How few examples can we get away with?
Delete one example and send again. Then another. Find the point where the classifier breaks. Ask the class which example turned out to be carrying the most weight, and why.
Step 3 · Teach a boundary
Write a genuinely borderline case as the incomplete example, something that could reasonably take two labels. In the bike shop pattern, “how much would it cost to fix a bent wheel” could be a repair or a question. Send it and see which way the LLM goes.
Now add one example that quietly settles the boundary, without a rule. For instance, add “thing: how much is a tune-up / label: question” and send the borderline case again.
- What did the one added example teach? Could you have written it as a rule? How long would the rule be?
Step 4 · Teach a procedure
Examples can teach more than labels. They can teach a way of working through a problem. Pick a kind of problem the class knows how to solve: diagnosing why a bike will not shift, deciding what to do when a group member disappears the week before a deadline. Write one worked example in each of two shapes, then give both the same new problem.
Shape A · without thinking
Here is Shape A filled in with a bike problem. The worked example shows how one problem was solved, and then the LLM has to work through a new one.
Complete the pattern. Output only the completion, with no explanation.
Understood.
problem: the bike will not shift into the largest rear gear action: check the tension on the shifter cable result: the cable is loose action: tighten the barrel adjuster half a turn and test each gear result: it shifts into every gear answer: the cable tension was too low; adjusted the barrel adjuster
problem: the rear brake lever pulls all the way to the handlebar before the bike slows action:
action: squeeze the lever and look at the brake pads against the rim result: the pads are worn nearly flat and barely touch the rim action: replace the pads and re-center the caliper result: the lever stops firmly about halfway answer: the pads were worn out; replaced them
Shape B · with thinking
Here is Shape B filled in with the same two problems. The only difference is the thought lines, which say why each action was chosen.
Complete the pattern. Output only the completion, with no explanation.
Understood.
problem: the bike will not shift into the largest rear gear thought: if it reaches every gear but the last one, the cable is probably not pulling far enough action: check the tension on the shifter cable result: the cable is loose thought: a loose cable explains it, and the barrel adjuster is the quickest fix action: tighten the barrel adjuster half a turn and test each gear result: it shifts into every gear answer: the cable tension was too low; adjusted the barrel adjuster
problem: the rear brake lever pulls all the way to the handlebar before the bike slows thought:
thought: a lever that goes all the way in means either the pads are too far from the rim or the cable has stretched, so look at the pads first because that is faster to check action: squeeze the lever and look at the brake pads against the rim result: the pads are worn nearly flat and barely touch the rim thought: worn pads explain the long pull, and new pads will also need the caliper centered action: replace the pads and re-center the caliper result: the lever stops firmly about halfway answer: the pads were worn out; replaced them
Put the two completions side by side. In Shape B the LLM said why it looked at the pads first and why it expected to re-center the caliper. Ask whether the thought line changed the quality of the actions chosen, or only how much was written.
Step 5 · Teach a language nobody speaks
Finish with the strangest one. Invent a notation with a hidden rule: arrows, symbols, a ~ for “not,” a ? for “maybe.” Write five translations from English into it. Then ask for a new sentence to be written in it, not decoded.
A made-up language
Here is that pattern sent as a conversation, and the completion that comes back.
Complete the pattern. Output only the completion, with no explanation.
Understood.
The dog runs. → dog → The dog does not run. → dog ~→ The cat sleeps. → cat ☾ Maybe the cat sleeps. → cat ☾? The dog runs, then sleeps. → dog → ☾
Maybe the dog does not sleep. →
dog ~☾?
Notice what was never written: a legend. The examples alone carry the grammar. The LLM had seen ~ before a verb symbol once and ? after one once, and it combined them correctly in a sentence it had never seen. Let students invent their own notation with a new hidden rule and see how few examples the LLM needs to speak it.
When examples want a little help
Sometimes pure examples wobble. The LLM invents a label that was not in the set, or misreads the boundary. The fix is rarely a paragraph. Notice that none of these are sentences either.
Declarations that are not prose
Here is a declaration in use. The bike shop classifier from Step 1 has no label for a message that is not a repair, a sale, or a question, so a thank-you note would force the LLM to guess. One labels line fixes that.
Complete the pattern. Output only the completion, with no explanation.
Understood.
labels: {repair, sale, question, other}
thing: my back brake squeals every time I stop label: repair
thing: do you have any used kids’ bikes under $150 label: sale
thing: what time do you close on Saturdays label: question
thing: the chain keeps slipping when I shift label: repair
thing: you guys did an amazing job on my bike, thank you label:
other
A set in braces, a dash list, a range written 0..10: these look instructive because the LLM has seen a million files that use them. Formats carry meaning the same way words do. In the conversation above, the LLM used a label that appeared in no example, because the labels line told it that label existed. When a pattern misbehaves, repair it with the smallest declaration that fixes it, and count the words you added.