RAG Architecture: Chunking, Embeddings, Vector Search

Retrieval-Augmented Generation grounds LLM output in your specific documents, drastically reducing hallucination and letting the model answer questions.

Retrieval-Augmented Generation: Grounding LLMs in Your Own Data

Retrieval-Augmented Generation (RAG) is the pattern where, at query time, relevant chunks of a source corpus are retrieved and inserted into the LLM prompt as context. The model then answers using that context rather than its training knowledge alone. It is how you get an LLM to accurately answer questions about your product docs, your customer's data, or last week's board memo. Getting RAG to production quality is 90% retrieval quality and 10% clever prompting — a fact many teams learn the hard way.

The pipeline stages

Ingest: source documents (PDFs, HTML, transcripts, database records) are cleaned, parsed, and split into chunks. Embed: each chunk is converted to a dense vector via an embedding model (OpenAI text-embedding-3, Cohere embed, Voyage, open-source BGE / E5). Store: vectors are written to a vector database (pgvector on Postgres, Pinecone, Weaviate, Qdrant, Turbopuffer) alongside metadata. Retrieve: at query time, the user question is embedded and the top-K nearest chunks are returned. Re-rank: a cross-encoder re-orders the top-K for better precision. Generate: retrieved chunks + question go into the LLM prompt; the model answers grounded in the context. Every stage has failure modes; instrument and measure each.

Chunking is where most RAG systems go wrong

The chunk is the unit of retrieval; get it wrong and no downstream cleverness rescues quality. Naive fixed-size chunking (500 tokens with 50 overlap) is a starting point, not a solution. Better: (1) Structural chunking — split on document structure (headings, sections, code blocks) so a chunk is semantically self-contained. (2) Sentence-aware — never split mid-sentence. (3) Include heading context in each chunk ('Section 3.2: Refund Policy — <chunk text>') so the retrieved snippet makes sense on its own. (4) Preserve tables as single chunks. (5) Store parent-child relationships — retrieve small chunks, feed larger parent windows to the LLM. Chunking is document-format-specific; PDFs, code, and chat transcripts all need different strategies.

Hybrid search beats vectors alone

Pure vector search is bad at proper nouns, part numbers, exact phrases, and rare tokens — the exact things users search for when they know what they want. Hybrid retrieval: run both vector search (semantic) and BM25/keyword search (lexical); combine results via reciprocal rank fusion or a weighted score. Postgres with pgvector + tsvector, or Elasticsearch/OpenSearch with a vector plugin, natively support this. Almost every serious RAG system in production is hybrid; vector-only is a demo pattern that plateaus.

Re-ranking cheaply

Retrieve top 20-50 candidates with hybrid search (fast, coarse). Re-rank them to top 5-10 with a cross-encoder (Cohere Rerank, Voyage Rerank, BGE-reranker) — slower per pair but massively more accurate because the model sees query and passage together. The uplift is often larger than any prompt-engineering trick. Cost: fractions of a cent per query at moderate volume. Skip re-ranking only if latency budget is under 500ms end-to-end.

Evaluation and iteration

Measure retrieval and generation separately. Retrieval: for a golden set of query-to-relevant-chunk pairs, compute recall@K and MRR — did the correct chunk end up in the top K? If not, no LLM prompt fixes it. Generation: given a good retrieval, does the LLM answer correctly and faithfully? Use LLM-as-judge with a rubric that penalizes unsupported claims. Common wins uncovered by systematic evals: change chunking strategy (20-40% recall lift), add hybrid search (10-20% lift), add re-ranking (10-20% precision lift), tune the prompt for citation ('quote the source passage before answering') to reduce hallucination.

Frequently asked questions

Do we need a specialized vector database?
For most startups under 10M chunks, no. pgvector on Postgres is production-grade, keeps your data in one place, and eliminates a moving part. Move to a specialized vector DB (Turbopuffer, Pinecone, Qdrant) when you need >10M vectors, sub-50ms latency at high QPS, or filtered search performance Postgres struggles with.
How does long-context (1M+ token) affect RAG?
Long context reduces some pressure but does not eliminate RAG. Costs and latency still scale with context; models still lose accuracy on the middle of very long contexts ('lost in the middle'); most importantly, dumping your entire corpus in-context does not work at real corpus sizes. RAG remains the pattern; long context lets you retrieve more generously (top-50 chunks instead of top-5).
When should we fine-tune instead of RAG?
Fine-tune for behavior (tone, format, task adherence); RAG for knowledge (facts, documents). If the answer changes when your docs change, that is a RAG problem, not a fine-tuning problem. Fine-tuning a model on your docs teaches it to memorize approximately — reliably worse than retrieval for factual accuracy.

Related fundraising guides (40)

Investor directory · Fundraising library · Articles A–Z · Company funding database