Answers are grounded in this paper and cite their sources.
Multimodal abstractive sentence summarization condenses a news sentence into a headline with the help of its accompanying photograph. Vision-guided models inject a global visual representation into the encoder, yet they treat the entities named in the source sentence as ordinary tokens, so the decoder receives little fine-grained grounding and the visual signal is easily swamped by the stronger linguistic signal. This paper presents an entity-guided contrastive framework built on BART. Knowledge-graph entities recognised in the source sentence are represented with pretrained TransE embeddings and encoded jointly with the text and with the image by two encoders that share parameters. A gated fusion module combines the two encoder outputs into a single memory for the decoder, and contrastive objectives align the resulting representations with each other and with the target-summary representation. On the 2,000 instances of the MMSS test set, and against a text-only variant trained from the same pretrained weights under the same recipe, the framework improves ROUGE-1 from 55.13 to 56.38, ROUGE-2 from 31.60 to 32.89 and ROUGE-L from 51.38 to 52.47, and a paired bootstrap places the ROUGE-1 gain outside the zero-difference interval. A diagnostic section locates the remaining headroom in the decoder rather than in the alignment: the contrastive representations do separate the two branches, but the decoder grants the visual block under two per cent of its cross-attention mass, which bounds what the alignment can contribute to the headline.
No takes yet. Share an insight, caveat, or question.
Zhou et al. (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: