When Reuse Repeats Mistakes: Answer Stability and Error Concentration in Large Language Model Semantic Caching

Semantic answer caching reduces inference cost by reusing previous responses but may repeatedly serve incorrect answers. Prior research primarily addresses cache matching, verification, and freshness, whereas this study investigates an error-attribution problem: grouping multiple request surfaces under shared source keys can concentrate incorrect answer exposure without proportionally increasing the total number of errors. We evaluated 500 category-balanced MMLU-Pro questions, six instruction-and-format variants, and two instruction-tuned models using fixed multiple-choice predictions. On 400 TEST questions, pairwise answer disagreement was 27.12% for Qwen2.5-3B and 26.90% for Mistral-7B. At a DEV-selected threshold of 0.96, TEST same-question matching achieved 98.59% precision and 99.97% recall. Semantic caching avoided 91.69–92.31% of modeled inference-token cost. Under Zipf demand, mean maximum cumulative wrong reuse per source key rose from 146.8 to 783.2 for Qwen and from 153.3 to 820.1 for Mistral. These approximately fivefold increases partly reflect different cache-key granularities: at the underlying-question level, maxima were 726.0 versus 783.2 and 745.8 versus 820.1, respectively. Net additional wrong labels relative to fresh computation were only 22.1 and 37.55 per 4800 requests. Exposure-aware refresh did not consistently improve accuracy or control-concentrated errors. Reliable cache evaluation should jointly consider correctness, computational cost, error exposure, and aggregation level.