TDI-SF: Trustworthy Dynamic Inference via Uncertainty-Gated Retrieval and Similarity-Gated Strict Fallback

Retrieval-time augmentation can correct hard test samples but may also introduce harmful interference when retrieved neighbors are unreliable. We propose TDI-SF (trustworthy dynamic inference via similarity-gated strict fallback), a safety-oriented dynamic inference strategy that intervenes only when needed and falls back to a frozen baseline when retrieval quality is insufficient. Uncertainty-gated selective retrieval triggers on a hard subset, defined by high entropy or low margin predictions (q=0.3), and similarity-gated fusion weights neighbor evidence by maximum similarity with a strict fallback threshold (alpha-mode=maxsim, min_maxsim). We evaluate on ImageNet-100 (ResNet-50) and CICIDS2017 (MLP) and report overall accuracy, hard-subset accuracy, calibration, negative flips, and risk–coverage behavior alongside efficiency. Comprehensive evaluation under both clean and degraded retrieval conditions demonstrates the value of each component. On ImageNet-100, TDI-SF improves hard-subset accuracy by 0.92% and overall accuracy by 0.30%, applying retrieval to only 32.6% of samples with 1.38 ms overhead per triggered sample. On CICIDS2017, the same mechanism yields +1.30% hard-subset gains with only 0.43 ms/hard overhead. These results show a simple, auditable recipe for safer retrieval-augmented inference across heterogeneous domains.