Bayesian BM25 - A Probabilistic Framework for Hybrid Text and Vector Search

Abstract This paper presents Bayesian BM25, a novel probabilistic framework that transforms traditional BM25 relevance scores into calibrated probability estimates, enabling principled integration of lexical and semantic search signals. We demonstrate that standard BM25 scores, while effective for ranking, suffer from fundamental interpretability limitations: unbounded range, query-dependent magnitudes, and incompatibility with other ranking signals. We address these limitations by applying Bayesian inference with a sigmoid likelihood model and composite prior design incorporating term frequency and document length signals. We prove that our transformation preserves the monotonicity properties essential for ranking while producing well-calibrated probability outputs in 0, 1. Key Contributions Probabilistic Calibration: Transforms unbounded BM25 scores into calibrated probabilities, solving the interpretability problem of raw scores. Principled Hybrid Search: Enables the combination of lexical (BM25) and semantic (Vector/HNSW) signals using probabilistic logic (AND/OR) without ad-hoc normalization or arbitrary weighting schemes. Monotonicity Preservation: Mathematically proves that the transformation preserves the ranking order of the original BM25, ensuring no loss in retrieval quality. Optimization Compatibility: Demonstrates compatibility with safe document pruning algorithms like WAND and BMW (Block-Max WAND) by maintaining upper bound properties. Numerical Stability: Introduces a log-space computation method to handle extreme probabilities and avoid underflow issues.