Federated Retrieval Over Heterogeneous Vector Stores: An Empirical Characterization with ImperaBench-CART

Natural language production retrieval-augmented systems increasingly span multiple vector databases with different embedding models. Existing vector clients assume a single store with a single embedder; cross-store federation is left to the application. We characterize the federation problem on ImperaBench-CART, a chunk-anchored benchmark of 354 queries over 5,933 chunks from 20 open-access biomedical papers, and release the benchmark, an evaluation harness, and the Impera library that the empirical findings motivate. Our central finding is that score-magnitude-based merging of per-source ranked outputs cannot reliably identify the source holding the answer. The holds-answer source's top-1 score is highest in only 34.5% of queries under heterogeneous embedders—statistically indistinguishable from the three-source chance rate of 1/3. A homogeneous-embedder control (three Qdrant shards under the same embedder) yields 33.9%: the phenomenon is not caused by embedder heterogeneity, but by partitioning combined with dense-embedder confidence behavior. This drives a design contract: Impera v1.6 returns per-source results in each store's native score space and defers cross-source ranking to the application via five pluggable merge strategies. We characterize each merger's recall on ImperaBench-CART with 95% bootstrap CIs and pairwise McNemar tests. Recall@10 across nine retrieval configurations (three native single-store baselines, three heterogeneous federated, three homogeneous federated) is 20%–30%; Recall@1 is 5%–9%. Federation matches the per-embedder recall ceiling and does not degrade retrieval. Relative to an otherwise identical homogeneous federation, heterogeneous federation pays a ~1.7× p50 latency cost and a ~4.5× throughput cost under load, attributable to the slowest participating embedder. The Tier-0 vs. Tier-1 paraphrase gap is +0 to +9 percentage points across configurations, isolating paraphrase robustness as a calibration target. The numerical findings are unflattering by construction: prior chunk-retrieval evaluations using embedder-defined ground truth report Recall@1 above 90% on equivalent setups, and the gap is attributable to the ground truth, not the system. The benchmark and its chunk-anchored qrels make future improvement falsifiable; the floor is now concrete.