RAG

RAG Pipeline Cheat Sheet

Chunking, embedding, retrieval and reranking — the decisions that matter

Retrieval-augmented generation in one page: the stages, the parameters worth tuning, sensible starting values, and the failure modes that make a RAG system look broken when it is only mis-tuned.

Advanced 2 min read 17 Entries Version 1.0 Sabir Updated 5 2
Download PDF Export Markdown Export HTML

The pipeline

Every RAG system is these six stages. Most quality problems are in stages 2 and 5.

Stage What happens Where it goes wrong
1. Ingest Load and clean source documents PDF extraction mangles tables and columns
2. Chunk Split into retrievable units Splitting mid-idea destroys meaning
3. Embed Turn chunks into vectors Mixing embedding models between index and query
4. Store Index vectors plus metadata No metadata means no filtering later
5. Retrieve Find candidates for the question Top-k too low, or pure vector search with no keyword fallback
6. Generate Answer using retrieved context No instruction to say "I do not know"

Starting values

Defaults that work for prose documents. Tune from here rather than from zero.

Parameter Start with Notes
chunk_size 600–800 tokens Smaller for dense reference, larger for narrative
chunk_overlap 10–15% of chunk_size Stops an idea being cut in half at a boundary
top_k 20 retrieved → 5 after rerank Retrieve wide, then narrow — not the other way around
hybrid_alpha 0.5 Balance of vector vs keyword; raise for jargon-heavy corpora
min_score unset at first Add only once you have measured a real threshold

Failure modes

  • Confidently wrong answers Add "answer only from the context; if it is not there, say so" and show sources.
  • Right document, wrong passage Your chunks are too large. Halve chunk_size and add a reranker.
  • Never finds obvious matches Pure vector search misses exact terms. Add keyword/BM25 hybrid retrieval.
  • Good in testing, poor in production Test questions matched your chunk boundaries. Test with real user phrasing.
  • Slow responses Rerank fewer candidates, cache embeddings, and stream the answer.

Code examples

Chunking with overlap

Sentence-aware splitting beats fixed character windows.

Python chunk.py Download 
def chunk(text, size=700, overlap=100):
    """Split on sentence ends so an idea is never cut in half."""
    import re
    sentences = re.split(r'(?<=[.!?])\s+', text.strip())
    chunks, current = [], ''

    for s in sentences:
        if len(current) + len(s) > size and current:
            chunks.append(current.strip())
            # carry the tail forward so context survives the boundary
            current = current[-overlap:] + ' '
        current += s + ' '

    if current.strip():
        chunks.append(current.strip())
    return chunks

Frequently asked questions

Do I need a vector database?
Not below roughly 10,000 chunks — an in-memory index or a Postgres extension is simpler and fast enough.
Is a bigger context window a replacement for RAG?
No. Cost scales with what you send, and retrieval quality still beats dumping everything into the prompt.

Resources

AI Workflows End-to-end automation patterns

Was this cheat sheet useful?

Comments

No comments yet — be the first.

Keep going

More cheat sheets

Browse all
Chunking Strategies Cheat Sheet RAG · 1 min read · 16 Entries Embeddings Cheat Sheet RAG · 1 min read · 10 Entries
Need a different cheat sheet? Tell us what you would like to see and we will build it — free.
Request a cheat sheet