Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40%

Standard RAG implementations often rely on fixed 512-token chunks and text-embedding-3-small, which fail in production for legal contracts, API docs, and customer tickets. These methods cause mid-sentence splits and high latency, often exceeding one second per query. To solve this, the retrieval layer was rebuilt from first principles. By moving away from basic semantic search toward a measured, tunable pipeline incorporating Bayesian Search, the team successfully cut latency by 40% and reached 95% recall@10.