Query-Aware Context Compression for Efficient Long-Context Language Model Inference

Query-aware context compression can reduce the amount of source text forwarded to a language model, but keyword ranking may discard evidence connected through intermediate entities. We present QAB-R, a training-free sentence selector combining BM25, query-identifier-constrained anchor selection, canonicalized entity matching, and at most two bridge-linked candidates under a fixed sentence budget. We first evaluate a single-bridge baseline on 300 synthetic single- and two-hop queries. We then execute 300 additional held-out stress cases covering paraphrases, competing entity notes, identifier formatting variation, and combined changes; a separate 60-case test extends source contexts to 58,368 whitespace-delimited words. At a 10% sentence budget, QAB-R preserved complete two-hop support in all 60 cases of each stress condition, whereas single-bridge retention fell to 0% under competing notes and format changes. For 1,024 fact pairs, the median QAB-R CPU preprocessing time was 66.3 ms, and the selected text contained 5,741 of 58,368 input words. The study reports evidence completeness, retained input length, and CPU preprocessing cost; downstream reader accuracy and GPU inference speed are distinct system-level outcomes.