Answers are grounded in this paper and cite their sources.
Benchmarking evaluation reveals contextual embedding bias predicts near-synonym confusion across language models, indicating isolated term geometry fails to foresee clinical substitution errors.
Near-synonym confusion —- where a semantically similar but functionally distinct term is substituted for another –– constitutes a clinically important failure case of LLMs deployed in medical and high-stakes contexts. In retrieval- augmented generation safety literature, it is commonly assumed that the cosine similarity between isolated term embeddings predicts the probability of such a confusion. We directly test this assumption. We present NEARSYNONYMBENCH, a benchmark of 120 domain- specific near-synonym pairs from medical, Indian cultural heritage and emergency blood resource domains, annotated with confusion probes and stakes levels. We test four sentence transformer architectures and three LLM families under forced choice conditions. Our main result is dissociation: isolated embedding geometry accounts for no variance in LLM confusion behavior at all (r = −0.005, p = 0.958), while contextual embedding bias, or difference in the proximity of the query probe to each candidate term, constitutes a statistically significant predictor (ρ = 0.299, p < 0.001), accounting for 45.3 times more variance. This effect is especially pronounced for high stakes pairs (r = 0.312, p = 0.018).We additionally identify terminological evasion — accurate description without term commitment, as an underreported failure mode accounting for 28.3% of open-ended responses. NEARSYNONYMBENCH and all code are released publicly.
No takes yet. Share an insight, caveat, or question.
Priyanka Asthana (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: