What question did this study set out to answer?

To test whether the document retrieval miss ceiling in AI agents can be reduced using ranked lexical retrieval methods like BM25 compared to exact search and rank fusion.

August 19, 2026Open Access

Finding More, Fusing Less: A Pre-Registered Locator Bake-off for AI Agents

View Full Paper

Ask AI

Bookmark

Authors

RARashid Azarang

Discussion

Member takes

Implication

Pre-registered comparative evaluation demonstrates that BM25 lexical retrieval outperforms exact search and rank fusion in AI agents, suggesting fusion defaults reduce localization accuracy.

Key Points

To test whether the document retrieval miss ceiling in AI agents can be reduced using ranked lexical retrieval methods like BM25 compared to exact search and rank fusion.
Evaluated four locator arms on a frozen 141-document corpus and a newly generated 115-question benchmark under an identical k=8 retrieval tool contract.
Tested pre-registered hypotheses comparing custom BM25 with language analyzers, hardened exact search, reciprocal rank fusion, and an off-the-shelf SQLite FTS5 control.
BM25 located the gold document in 93.0% of queries compared to 71.3% for exact search (+21.7 percentage points, McNemar p < 1e-5).
Reciprocal rank fusion achieved 85.2% localization (+13.9 pp over exact search, p = 0.00014), trailing standalone BM25 by 7.8 pp (p = 0.0225) and rejecting the hypothesis that fusion never harms accuracy.
SQLite FTS5 reached an 89.6% localization rate with no statistically significant difference from custom BM25, yielding a net downstream accuracy increase of +5.2 pp.

Cite This Study

Rashid Azarang (2026) studied this question.

synapsesocial.com/papers/6a85634f03308d306e2d666f https://doi.org/10.5281/zenodo.21969900

View Full Paper

Ask AI

Bookmark

Implication

Pre-registered comparative evaluation demonstrates that BM25 lexical retrieval outperforms exact search and rank fusion in AI agents, suggesting fusion defaults reduce localization accuracy.

Finding More, Fusing Less: A Pre-Registered Locator Bake-off for AI Agents

Discussion

Member takes

Implication

Key Points

Cite This Study

Finding More, Fusing Less: A Pre-Registered Locator Bake-off for AI Agents

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider