Embeddings and Similarity

You want an agent that answers questions about your material — handbooks, tickets, design notes, contracts. Not generic web knowledge. Your files.

Say you have thousands of documents.

Someone asks: “How do we reset a contractor’s laptop?”

Somewhere in that pile, the answer exists. Your agent must find it and reply in plain language.

You cannot send everything

A language model only sees what you put in this request.

If you paste all thousands of documents into every request:

So the job is not “give the model the whole library.”
The job is: find a few useful passages for this question, then ask the model.

  thousands of documents
           │
           │  ✗ cannot all go in one request
           │
           ▼
  find a small set that
  matches THIS question
           │
           ▼
  send those + the question
  to the chat model

That “find a small set” step is the heart of this story. This chapter teaches the idea that makes meaning-based find possible. The next chapter wires find → answer end to end.

First try: search by words

A simple approach: look for documents that share words with the question.

Question: “How do we reset a contractor’s laptop?”

A note says: “Procedure to wipe a loaner computer before return.”

Same idea. Different words. Keyword search often misses it.

People read meaning. Your program needs a way to score meaning too.

Embeddings: meaning as numbers

An embedding is a long list of numbers that stands for what a piece of text is about.

Another word for that list is vector. Same idea.

You do not invent the numbers. You send text to an embedding model — a specialized model whose only job is:

text in  →  list of numbers out

It is not your chat model. Chat models write replies. Embedding models place meaning on a map.

"Reset a contractor laptop"
  →  [0.12, -0.04, 0.33, …]

"Wipe a loaner computer before return"
  →  [0.11, -0.05, 0.31, …]   ← nearby numbers

"Python is a programming language"
  →  [-0.22, 0.41, -0.09, …]  ← far away

Similar meaning → similar lists. Unrelated meaning → different lists.

Once every passage is a list of numbers, “find useful docs” becomes: embed the question, then keep the passages whose numbers sit closest.

A tiny map you can hold in your head

Real embeddings have hundreds or thousands of numbers. The math is the same with two, so we can draw it.

Pretend these toy vectors (made-up for teaching):

  y
  │        Python (-2, 3)
  │
  │
  │   wipe loaner (3.6, 1.1)
  │  reset laptop (4, 1)
  │
  └────────────────── x

The question and the IT note sit near each other. The Python line sits elsewhere. Your program can keep the near ones and ignore the far ones — even among thousands of points.

Similarity: a score for “how near”

Your program needs a number, not a drawing.

The usual score for text embeddings is cosine similarity. In plain words: do two lists point the same direction?

You almost never care about the exact digits. You care about order:

• related passages → higher score
• unrelated passages → lower score

Worked idea

Question: “How do we reset a contractor’s laptop?”

Stored passages (among thousands):

1. “Wipe a loaner computer before return.”
2. “Renew the Python license by Friday.”

Embed the question and both passages. Passage 1 scores high. Passage 2 scores low. Keep passage 1 for the model.

That is meaning search over a library you cannot paste whole.

What a call looks like

Same family as your other API calls: send text, get structured data back. Here the data is numbers, not a chat reply.

You send (idea):

{
  "model": "an-embedding-model",
  "texts": [
    "How do we reset a contractor laptop?",
    "Wipe a loaner computer before return."
  ]
}

You get (idea):

{
  "embeddings": [
    { "values": [0.12, -0.04, 0.33, "…many more…"] },
    { "values": [0.11, -0.05, 0.31, "…many more…"] }
  ]
}

One list of numbers per input string, same order.

Then your code compares two lists with cosine similarity. That step is ordinary math on your machine. You do not need another model call just to score two vectors you already have.

In practice you embed documents once when you index them, store the numbers, and at question time you mostly embed the question and compare.

Where this sits in the agent story

You already know how to call a model and how to give it tools.

For a document agent, the missing piece is a smart lookup:

  person asks about YOUR docs
           │
           ▼
  embed the question
           │
           ▼
  compare to stored passage embeddings
           │
           ▼
  keep the closest few passages
           │
           ▼
  send passages + question
  to the chat model  →  answer

This chapter is the middle of that path: embed and compare.

Turning “closest passages + question → grounded answer,” including how you split long files into passages, is the next chapter: retrieval augmented generation (RAG).

Two chapters, one story:

Rules that save pain

When things go wrong

Words to keep

See it in Code

The Code panel embeds a few short sentences and prints similarity scores. Related pairs should beat unrelated ones — the same ranking idea you would use across thousands of passages.