Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40%

The team optimized RAG at scale by replacing standard semantic search with a measured, tunable retrieval pipeline. By moving beyond fixed 512-token chunks and text-embedding-3-small, they addressed production issues in legal contracts, API docs, and customer tickets. This first-principles rebuild utilized Bayesian Search to cut latency by 40% and achieve 95% recall@10. The approach solves common production bottlenecks where standard chunking methods often split critical clauses or drown signals in noise.