A Conversation Is One Big Prompt

Overview

In order to use computational thought, we send prompts to a large language model. Prompts are our inputs to the LLM — the text that gets it to apply its computational thought. That is: prompts are how we program computational thought.

It is very important to understand the relationship between prompts and the conversations we are used to having with LLMs — and, most importantly, to understand that a conversation actually is one big prompt. All of our interactions with an LLM are structured around conversation.

An agent emerges from a prompt with a specific conversational structure

The reason the relationship between prompts and conversations deserves this much attention: an agent is a prompt with certain properties — and a little machinery around it. To build agents, we must first master exactly what a prompt is, and develop an intuitive understanding of its properties and how it impacts an LLM.

A prompt is the text we send to the Large Language Model (LLM)

Start with the simplest way to think about it: a prompt is some text that is sent to the LLM. The LLM responds with some text.

Prompts are conversations at a point in time

Now think of a prompt as a conversation at a point in time. Humans already know how to write conversations down — we do it in documents, as scripts:

If I asked you to read this and tell me who is doing what, you could — and much more. A really great script tells you who the actors are. It gives you all kinds of clues about their state of mind, their actions, their history. Three lines in, you know Kari is pressed and prepared, Ola is skeptical and settled, they have had this argument before, and today something is different: the van is loaded. You could make a good prediction about what happens next.

You can answer those questions because you read the scene — it carries excellent context for understanding the situation, and that context is what lets you predict how things will evolve. A really good prompt operates the same way. When we say a prompt provides "context," think of it intuitively, the way you just read that script: it builds up an understanding of a situation — the actors, what has happened, what is at stake — and that understanding shapes what comes next.

A chat conversation with an LLM is one big prompt — despite what it looks like

Here is a short conversation with an LLM:

What is the prompt?

In the chat window, it looks like we sent "What should I pack?" and got an answer. But that is not what was sent. What was sent was the entire conversation, written down like a script:

This whole document is the prompt. Every time we add a message, it appears that we sent just that message — but we are actually sending the entire conversation to the LLM, and it responds to all of it.

Connect this back to where we started. A prompt is some text sent to the LLM, and the LLM responds with some text. The text being sent is this whole script. And in the same way you read the script about Kari and Ola and understood the actors, the situation, and the history — and could predict what came next — the same thing is happening with this prompt. The conversation embodies the situation: who is talking, what they want, what has already been said. That is how the reply to "What should I pack?" knows the place and the month, even though those four words mention neither.

For the LLM, that conversation is all that exists. It has no memory and no connection to the outside world. If something is not in the conversation, it is not anywhere.

The copy-and-paste test

You can check this yourself. Below is the same conversation, stopped just before the packing answer. Click Raw on it, copy all the text, paste it into a brand-new session with the LLM of your choice, and see what comes back:

Prompts program the LLM to apply computational thought to a problem

A prompt is how we program the LLM to apply computational thought to a problem. Instead of writing code that specifies each step of a computation, we use conversation to configure what the LLM should think about, what information it should consider, what kind of analysis it should perform, and what it should produce as a result.

For example, imagine we have recorded a meeting about an upcoming trip. We don't just want the LLM to summarize the meeting — we want it to apply a particular pattern of computational thought to it: gap analysis.

Look at the final user message: "Perform a gap analysis." On its own, those words say nothing about what to analyze — no transcript, no trip, no plan. The computation was not programmed by that one message. It was programmed by the entire conversation: the first message put the transcript into the conversation, the assistant's reply established what the material is, and only then did "perform a gap analysis" name the operation. To respond, the LLM has to read the whole conversation — and the whole conversation is what tells it what it is operating on and what operation to perform.

This is the central idea: we use conversation as a programming language for computational thought. We construct a conversation that configures the LLM's reasoning, give it a problem, and ask it to apply that reasoning to produce a useful result.