RAG (Retrieval-Augmented Generation) is one of the most widely deployed technical architectures in enterprise AI applications today. The core idea: before generating a response, retrieve relevant document chunks from an external knowledge base and provide them as context to the model — grounding answers in real documents rather than training data “memory.” This approach effectively addresses LLMs’ two major pain points: hallucination and knowledge cutoff.
## Standard RAG Pipeline
**Document processing (offline)**:
1. **Chunking**: split long documents into embedding-suitable blocks (typically 256–1024 tokens); chunking strategy (fixed-size, semantic, recursive) directly affects retrieval quality
2. **Embedding**: convert each chunk into high-dimensional vectors using embedding models (OpenAI text-embedding-3, BGE, Nomic, etc.)
3. **Indexing**: store vectors in a vector database (Chroma, Pinecone, Weaviate, pgvector, etc.)
**Retrieval-generation (online)**:
1. User question → question vectorization
2. Vector similarity search → retrieve Top-K relevant chunks
3. Assemble chunks + user question into prompt
4. LLM generates final response with source citations
## Advanced RAG Techniques
**Hybrid search**: combine vector semantic retrieval with BM25 keyword retrieval, fusing rankings with RRF (Reciprocal Rank Fusion). Pure vector retrieval can miss exact matches for proper nouns; hybrid search is more consistently reliable.
**Reranking**: after retrieving Top-K chunks, use a cross-encoder model to rerank chunks, selecting the truly most relevant Top-N. Cohere Rerank and BGE-Reranker are common choices.
**GraphRAG (Microsoft)**: constructs document content as a knowledge graph, using graph structure to capture entity relationships. Suited for complex queries requiring reasoning across document relationships.
**Self-RAG**: the model autonomously decides when and what to retrieve rather than forcing retrieval on all queries, reducing unnecessary retrieval interference on generation quality.
See [Local LLM Deployment](https://sunqi.org/local-llm-deployment-en/) and [LlamaIndex documentation](https://docs.llamaindex.ai/).
