Embeddings & Vector Databases
What are embeddings?
An embedding is a dense vector that captures meaning: a model maps text (or images) into a space where similar things sit close together. “dog” lands near “puppy” and far from “car.”
Meaning becomes geometry: related concepts cluster. A query embeds to a point, and its nearest neighbors are the most semantically similar items.
- “Embeddings vs one-hot?” — One-hot vectors are sparse and meaningless; embeddings are dense and capture similarity. (See Word Embeddings.)
How does semantic similarity search work?
Embed every document once and store the vectors. At query time, embed the query and return the nearest vectors by distance — matching on meaning, not keywords, so “how to reset my password” finds “account recovery steps.”
- “Where is this used?” — Retrieval for RAG, recommendations, deduplication, and semantic search.
- “Keyword vs semantic search?” — Keyword matches exact terms; semantic matches meaning. Hybrid search combines both.
Why cosine similarity?
Cosine measures the angle between vectors, ignoring magnitude — so it compares direction (meaning) rather than length (which often just reflects text size). 1 = identical direction, 0 = unrelated, −1 = opposite.
- “Cosine vs Euclidean?” — On normalized vectors they rank identically; cosine is the convention for text embeddings.
- “Dot product?” — Equals cosine when vectors are unit-normalized, and it's cheaper to compute.
How do vector databases scale?
Comparing a query against millions of vectors exactly is too slow, so vector DBs use approximate nearest neighbor (ANN) indexes (e.g. HNSW) that trade a tiny bit of recall for huge speedups.
ANN indexing, metadata filtering, scalable storage, and updates — Pinecone, Weaviate, Milvus, pgvector (FAISS is an ANN library, not a full DB). A vector DB is the storage layer behind production RAG.
- “What is HNSW?” — A graph-based ANN index giving fast, high-recall nearest-neighbor search.
- “Recall vs latency?” — ANN params tune the tradeoff: higher recall costs more time/memory.