Rethinking Metaphor Evaluation: Aptness Judgments as a Cognitive Probe for Language Models

Metaphor is a pervasive rhetorical resource and a core cognitive mechanism for cross-domain conceptual mapping. We define metaphorical aptness as the extent to which a metaphorical expression captures the core attributes of the target concept, modeled as a continuous measure of how well source properties map onto the target in context. Although large language models (LLMs) have recently achieved strong results on metaphor detection, it remains unclear whether these gains reflect surface cue learning or human-like semantic integration. We propose an aptness-based continuum evaluation framework that extends binary identification into a finer-grained test of cognitive alignment. Experiments on the VU Amsterdam Metaphor Corpus (VUA), the largest English metaphor dataset, across representative LLMs (e.g., GPT-4, LLaMA-3) reveal a consistent gradient: performance is substantially higher on high-aptness metaphors than on low-aptness ones, suggesting models are more reliable when the mapping preserves central target properties but struggle when those attributes are weakly captured. We further replace binary labels with aptness annotations as supervision and show that a baseline trained under this paradigm improves metaphor detection over prior state-of-the-art systems. Overall, aptness provides an informative alignment signal and a principled basis for metaphor comprehension evaluation grounded in how metaphors convey core target attributes.