Answers are grounded in this paper and cite their sources.
Demonstrates competitive performance in grounded multimodal named entity recognition, suggesting potential for practical applications.
Grounded Multimodal Named Entity Recognition (GMNER) aims to extract named entities, their types, and corresponding visual objects from image-text pairs. However, existing GMNER methods rely on costly multimodal annotations, limiting their scalability in real applications. To address this issue, we propose a visual annotation-free framework that leverages text-only NER data and a Zero-shot Entity Visual Grounding (ZeroEVG) approach. ZeroEVG consists of three modules: (1) Candidate Object Generation, which pre-selects visual object candidates; (2) Entity-Object Matching, which determines whether an entity has a visual presence; and (3) Entity Visual Localization, which employs a variant of GradCAM to identify bounding boxes for groundable entities. Experimental results on two benchmark datasets show that our visual annotation-free framework achieves competitive performance with fully supervised multimodal approaches, and even surpasses some of them under the same backbone on both GMNER and EVG tasks.
No takes yet. Share an insight, caveat, or question.
Yang et al. (2025) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: