Use Ctrl+P (or Cmd+P) to save as PDF. Back to paper
Every vector database needs a way to turn text into numbers. Most engineers reach for the nearest embedding model — all-MiniLM-L6-v2, text-embedding-3-small, nomic-embed-text — run it once at setup, and treat it as infrastructure. They never touch it again.
The embedding model is not infrastructure. It is the floor of your retrieval quality.
A bi-encoder embedding model encodes a document into a single fixed-length vector. At query time, it encodes the query the same way. Retrieval is a nearest-neighbor search in that vector space — fast, scalable, and entirely dependent on the model's ability to compress meaning into a fixed number of dimensions.
The model determines what the vector space can express. A model trained on general web text produces a general-purpose space. A model trained on legal corpora produces a space where legal distinctions compress well. No embedding model is neutral.
The MTEB benchmark (Muennighoff et al., 2023) showed this concretely: state-of-the-art models vary from 512 to 32,768 token max lengths, from 256 to 4,096 dimensions, and from BERT-scale to 7B-parameter LLMs as backbones. The variation in retrieval performance across task types is not random — it follows the training distribution of the model.
Most embedding models output a fixed-length vector. You accept it, store it, and search it. But dimensionality is not free — it drives storage costs, index build time, and search latency directly.
Higher dimensions do not always mean better retrieval. They mean more storage, slower approximate nearest-neighbor search, and a higher probability of the curse of dimensionality at small corpus sizes. The relationship between dimension count and retrieval quality is not monotonic. It depends on your corpus, your queries, and your task.
Kusupati et al. (2022) formalized this problem and proposed a solution: Matryoshka Representation Learning (MRL). The key insight is that a high-dimensional embedding encodes its coarsest, most important structure first. MRL exploits this by training a single model to produce embeddings where any prefix of the vector is a valid, useful representation at lower dimensionality.
The result: a 1536-dimension MRL embedding can be truncated to 256 dimensions, retaining most of the retrieval quality at one-sixth the storage cost. The truncation is not an approximation — it is what the model was trained to support.
OpenAI's text-embedding-3-small (up to 1536 dimensions) and text-embedding-3-large (up to 3072 dimensions) both implement MRL. Nomic's nomic-embed-text-v1.5 does the same and ships as a fully open-weight model. This is no longer an academic technique — it is in production embedders you can run today.
Accepting a single vector per document has a hard ceiling: the entire document, however long, must compress into a fixed point in a shared query-document space. For short, precise documents, this is fine. For long documents with multiple topics, the compression loses information.
ColBERT (Khattab & Zaharia, 2020) proposed a different structure: keep one vector per token, and score query-document relevance using a late-interaction MaxSim operation. It approximates cross-encoder quality at bi-encoder inference cost. The trade-off is index size — storing per-token vectors is expensive. This makes ColBERT a second-stage scorer over a bi-encoder's top-k candidates, not a first-stage retriever over a full corpus.
The practical architecture: bi-encoder first-stage retrieval over the full index → ColBERT or cross-encoder reranking over the top 50–100 candidates → LLM. The embedding model choice governs stage one. Stage one governs the recall ceiling for every stage that follows.
When you pick an embedding model, you are deciding:
1. Domain fit. General-purpose models underperform on specialized corpora. If your documents are medical, legal, or code-heavy, benchmark a domain-specific model against your actual data — not MTEB.
2. Dimensionality budget. If your model supports MRL, benchmark at 256, 512, and 1024 dimensions against your retrieval task before committing to the default 1536 or 3072. You may be paying 6x the storage for marginal quality gains.
3. Recall vs. precision. A bi-encoder maximizes recall at scale. If your task requires precision, a reranker is mandatory — not optional.
The default is a starting point. The decision is yours.