Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40%

Standard RAG implementations often rely on fixed 512-token chunks and text-embedding-3-small, which fail in production for legal contracts, API docs, and customer tickets. These methods cause signal loss and high latency, often exceeding one second per query. To solve this, the retrieval layer was rebuilt from first principles. By moving away from basic semantic search toward a measured, tunable pipeline using Bayesian Search, the system achieved 95% recall@10 and reduced latency by 40%.