Searching by Meaning

A debt from earlier

When we designed tool results, we left a wound open. The agent needed the night-delivery rules; we gave it search; watch again what happened:

⟨tool_use⟩
search_document(doc = "doc-77", pattern = "night deliver")

⟨tool_result⟩
0 matches.

⟨tool_use⟩
search_document(doc = "doc-77", pattern = "night")

⟨tool_result⟩
2 matches: p. 12 "overnight mooring"; p. 98 "night watch staffing".

Meaning has a geography

Take a document everyone in a university knows: the undergraduate catalog — hundreds of pages of majors, requirements, prerequisites, and policies. Pull out a few statements and ask which ones are similar in meaning:

That judgment — same meaning, different words — is what we need the machine to make. An embedding is the device that makes it: a machine that reads a piece of text and produces its address — a point in a space arranged so that one rule holds everywhere:

Near in MEANING  =  near in SPACE.

Not similar spelling. Not shared words.
Similar meaning.

We can't draw the real space — it has thousands of dimensions — but we can look at its shadow in two, and even the shadow shows the structure:

The requirement statements cluster; the campus-life statements cluster elsewhere; A and B sit nearly on top of each other despite sharing no words. (This is the apple again, from the outside: the word connects to a rich learned representation, and the embedding gives that interior structure an address. It even crosses languages — eple lands beside apple on this map, because Norwegian text about eple behaves like English text about apples. Nobody told the space that. It was drawn by reading.)

What is the meaning of an entire catalog?

Now the question that makes this practical. The embedding machine will accept any text you hand it — a sentence, a page, the whole catalog — and hand back one point. So: embed the entire catalog. What does that point mean?

It isn't literally an average, but it behaves like one: one address must stand for everything in the text, and the more the text says, the less the address can say. Detail washes out. And mixtures make it worse — the catalog contains statements that pull in opposite directions:

Recognize what embedding-something-large is: it's abstraction — detail traded away for one compact representation — except nobody chose what to keep. Embedding a whole document is abstraction without an author. If we want sharp addresses, we must hand the machine sharp pieces.

Tearing the pages

So we cut the catalog into pieces and embed each piece — this is called chunking, and the honest physical picture is tearing: take each page and tear it into strips, and every strip gets its own address. But here is the trap: the tearing is usually done by ordinary code — every N lines, every paragraph break — and code doesn't know where the meaning bends. Watch a real page get torn:

Now embed the strips and ask the obvious question:

⟨tool_use⟩
search_by_meaning(query = "math courses required for the computer
science major", top = 3)

⟨tool_result⟩
1. strip 118 — Mathematics major: core requirements…
2. strip 41 — CS department history… "REQUIRED COURSES —
   COMPUTER SCIENCE MAJOR:" [list follows on next strip]
3. strip 87 — Math minor: course list…

A chunk's address is computed from the chunk alone.
Whatever context the tear separates from it — the heading,
the section, the "this is about X" — is subtracted from
its meaning.

Tear along the meaning, or the meaning tears.

The size of the tear

And tearing has a second dial: how big? We've now seen both ends fail:

The same Goldilocks we met designing results, now on the ingestion side — and the same lesson: the right size is defined by the questions that will be asked, not by a number of lines.

Smarter tearing

If the problem is that code doesn't understand where meaning bends — we have a machine that does. Two ingestion patterns, both built from self-dialogue:

Let a mind place the tears. Before cutting, send each page through a disposable prompt: "Return the line numbers where this page should be split so each piece states one complete idea with its heading and context." The LLM reads page 212 and tears above the heading, keeping "REQUIRED COURSES — COMPUTER SCIENCE" attached to its list — and joining the list's continuation from page 213 to the same piece. Pay a prompt per page, once, at ingestion; every future search benefits.

Cast each piece as the questions it answers. An even stronger move. A chunk and the query that seeks it are different kinds of text — one is a list, the other is a question — and different kinds of text sit in slightly different neighborhoods. So, at ingestion, ask a disposable prompt: "What questions could this piece answer?" — and embed those:

Notice what this is: at ingestion time, a mind imagined the future conversations this piece would matter to — and filed it where those conversations will look. The librarian shelving books by the questions readers will ask, not by the color of the covers.

Smarter asking

The same intelligence helps on the query side. A real task is usually not one clean question — and embedding the whole task has the same blur problem as embedding the whole catalog. So let the agent decompose before it searches, the way chain of thought decomposes before it answers:

⟨tool_use⟩
search_by_meaning(query = "courses required for the computer
science major", top = 2)

⟨tool_use⟩
search_by_meaning(query = "transfer credit between majors —
courses counting toward a new major", top = 2)

⟨tool_use⟩
search_by_meaning(query = "maximum course load per semester", top
= 2)

⟨tool_result⟩
[three result sets: the CS requirements piece; the STAT 3011
counting rules for both majors; the overload policy…]

What the map doesn't know

Before we build on this, the honest limits — because the map has exactly one skill, aboutness, and several things that look like aboutness are not:

So the retrieval ladder from earlier doesn't get replaced — it gets its missing rung:

Each rung up costs more and understands more. A well-equipped agent holds several rungs at once — and choosing the rung per question is part of its job.

Meaning has a geography, and an embedding is a text's address
in it. But the address is only as sharp as the piece:
tear along the meaning, size the pieces to the questions,
and let a mind help on both sides —
filing the pieces, and phrasing the search.