Vector databases are central to how modern AI applications store and retrieve information — they power the RAG (Retrieval-Augmented Generation) systems that make AI know about specific documents. Here is an accessible explanation of how they work and when to use them.
Why Regular Databases Don’t Work for AI Search
Keyword search (SQL LIKE queries, Elasticsearch) finds exact or fuzzy text matches. Vector search finds semantic similarity — “cat” and “feline” would return similar results because they are semantically related, not because they share letters. This matters because LLM applications need to find relevant context even when the query and the document use different words. A user asking “how do I reset my password?” should retrieve a document titled “account recovery procedures” — keyword search misses this; vector search catches it.
How Embeddings Work
An embedding model (text-embedding-3-small from OpenAI, Voyage from Anthropic, or open-source alternatives like BGE) converts text into a vector — a list of numbers (typically 768–3072 dimensions) that captures the semantic meaning. Texts with similar meanings produce similar vectors (close in multidimensional space). A vector database stores these vectors and can efficiently find the most similar ones to a query vector — this is approximate nearest-neighbour search (ANN).
Options Available
Pinecone: managed, easy to use, free tier available. Weaviate: open source with hosted option, good for hybrid (vector + keyword) search. Qdrant: open source, excellent performance, good self-hosted option. Chroma: lightweight, great for development and small scale. PostgreSQL + pgvector: if you already use PostgreSQL, pgvector extension adds vector search — avoiding a separate database for smaller scales.
The Minimal RAG Implementation
1. Chunk your documents (split into ~500 token pieces). 2. Generate embeddings for each chunk. 3. Store embeddings in vector database. 4. On user query: embed the query, find the 5 most similar chunks, include those chunks in your LLM prompt as context. 5. The LLM answers based on retrieved context. This is the basic pattern; production systems add reranking, metadata filtering, and hybrid search.
