A Conversation Is One Big Prompt

A prompt is text in, text out

An agent is a prompt with certain properties — and a little machinery around it. So it is very important to understand exactly what a prompt is, and to develop an intuitive understanding of its properties and how it impacts an LLM.

Start with the simplest way to think about it: a prompt is some text that is sent to the LLM. The LLM responds with some text.

That's the whole transaction. Text went in, text came back:

Prompts are conversations at a point in time

Now think of a prompt as a conversation at a point in time. Humans already know how to write conversations down — we do it in documents, as scripts:

If I asked you to read this and tell me who is doing what, you could — and much more. A really great script tells you who the actors are. It gives you all kinds of clues about their state of mind, their actions, their history. Three lines in, you know Kari is pressed and prepared, Ola is skeptical and settled, they have had this argument before, and today something is different: the van is loaded. You could make a good prediction about what happens next.

You can answer those questions because you read the scene — it carries excellent context for understanding the situation, and that context is what lets you predict how things will evolve. A really good prompt operates the same way. When we say a prompt provides "context," think of it intuitively, the way you just read that script: it builds up an understanding of a situation — the actors, what has happened, what is at stake — and that understanding shapes what comes next.

A chat conversation with an LLM is one big prompt — despite what it looks like

Here is a short conversation with an LLM:

What is the prompt?

In the chat window, it looks like we sent "What should I pack?" and got an answer. But that is not what was sent. What was sent was the entire conversation, written down like a script:

This whole document is the prompt. Every time we add a message, it appears that we sent just that message — but we are actually sending the entire conversation to the LLM, and it responds to all of it.

Connect this back to where we started. A prompt is some text sent to the LLM, and the LLM responds with some text. The text being sent is this whole script. And in the same way you read the script about Kari and Ola and understood the actors, the situation, and the history — and could predict what came next — the same thing is happening with this prompt. The conversation embodies the situation: who is talking, what they want, what has already been said. That is how the reply to "What should I pack?" knows the place and the month, even though those four words mention neither.

For the LLM, that conversation is all that exists. It has no memory and no connection to the outside world. If something is not in the conversation, it is not anywhere.

The copy-and-paste test

You can check this yourself. Below is the same conversation, stopped just before the packing answer. Click Raw on it, copy all the text, paste it into a brand-new session with the LLM of your choice, and see what comes back: