What Is an Embedding?

An embedding is a list of numbers — a vector — that represents the meaning of a piece of text. A sentence like 'The dog chased the ball' might become a vector of 1536 numbers. Those numbers aren't arbitrary: they're learned by a model trained to put texts with similar meanings close together in that high-dimensional space.

The practical power of this is that you can measure similarity between any two pieces of text by comparing their vectors — without any keyword overlap. 'Automobile' and 'car' produce similar vectors; 'car' and 'democracy' produce distant ones. This lets you find semantically related content that exact search would miss.

Embeddings are produced by embedding models — a specialized type of model that takes text in and outputs a fixed-size vector. They're separate from, and often smaller than, the generation models (like Claude or GPT-4) used for producing text. OpenAI's text-embedding-3-small, Cohere's Embed v3, and open-source models like BGE and E5 are common choices.