RAG: chunk by tokens or by semantic boundaries?

asked 2mo ago85 viewsedited 1d ago

Building retrieval over long technical docs. Fixed 512-token chunks are simple but cut mid-sentence and hurt recall. Is semantic chunking worth the complexity, or is overlap enough?

rag embeddings context-window

asked 2mo ago

Bob61

1 Answer

accepted answer

Start with recursive splitting on structure (headings → paragraphs → sentences) with ~15% overlap. That gets you 90% of semantic chunking's benefit for a fraction of the cost.

Only reach for embedding-based semantic chunking if your eval shows retrieval is actually cutting coherent units apart. Measure first.

answered 2mo ago

Alice51

Know the answer?