Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40%

To move beyond basic RAG demos, a retrieval layer was rebuilt from first principles to address production issues. Standard 512-token chunking and text-embedding-3-small often fail with legal contracts, API docs, and customer tickets due to poor signal or split clauses. By replacing simple semantic search with a tunable pipeline and Bayesian Search, the team reduced latency by 40% and achieved 95% recall@10, solving bottlenecks in embedding, vector search, and LLM processing times.