May 24, 2024Open Access

VDGD: Mitigating LVLM Hallucinations in Cognitive Prompts by Bridging the Visual Perception Gap

View Full Paper

Ask AI

Bookmark

Authors

SGSreyan GhoshIndraprastha Institute of Information Technology Delhi CEChandra Kiran Reddy EvuruAdobe Systems (United States)SKSonal KumarUniversity of Maryland, College Park

Discussion

Member takes

Implication

Computational study demonstrates that visual description grounded decoding reduces hallucinations in large vision-language models, highlighting the need to bridge the visual perception gap.

Key Points

Visual description grounded decoding mitigates hallucinations during cognitive prompts, addressing the core visual perception gap where models recognize items without true understanding.
Benchmark on multiple large vision-language models shows that sampling candidates via kl-divergence to image descriptions consistently reduces errors across complex cognitive tasks.
Evaluation using the novel vallu benchmark highlights persisting cognitive limitations, providing a standardized environment to measure visual perception in vision-language models.

Cite This Study

Ghosh et al. (2024) studied this question.

synapsesocial.com/papers/68e6897cb6db643587611541 https://doi.org/10.48550/arxiv.2405.15683

View Full Paper

Ask AI

Bookmark

Implication

Computational study demonstrates that visual description grounded decoding reduces hallucinations in large vision-language models, highlighting the need to bridge the visual perception gap.

VDGD: Mitigating LVLM Hallucinations in Cognitive Prompts by Bridging the Visual Perception Gap

Discussion

Member takes

Implication

Key Points

Cite This Study

VDGD: Mitigating LVLM Hallucinations in Cognitive Prompts by Bridging the Visual Perception Gap

Discussion

Member takes

Implication

Key Points

Cite This Study

Also Consider

Also Consider