February 11, 2024Open Access

A Benchmark for Multi-modal Foundation Models on Low-level Vision: from Single Images to Pairs

View Full Paper

Ask AI

Bookmark

Most multimodal large language models demonstrate basic low-level visual perception on single images, but only GPT-4V achieves superior accuracy in pairwise comparison tasks.
Evaluations across 24 models and 7 image quality assessment datasets reveal that paired evaluations better emulate human perception than isolated single-image quality scoring.
Benchmark on LLVisionQA+ and LLDescribe+ datasets standardizes visual question answering and text descriptions, highlighting key pathways to advance low-level visual foundation models.

Discussion

No takes yet. Share an insight, caveat, or question.

Implication

Benchmark evaluation demonstrates low-level visual perception gaps in multimodal large language models, indicating that only advanced models excel at pairwise comparisons.

Cite This Study

Zhang et al. (2024) studied this question.

synapsesocial.com/papers/68e79abcb6db64358770a712 https://doi.org/10.48550/arxiv.2402.07116

View Full Paper

Ask AI

Bookmark

Implication

Benchmark evaluation demonstrates low-level visual perception gaps in multimodal large language models, indicating that only advanced models excel at pairwise comparisons.

A Benchmark for Multi-modal Foundation Models on Low-level Vision: from Single Images to Pairs

Key Points

Discussion

Implication

Cite This Study

A Benchmark for Multi-modal Foundation Models on Low-level Vision: from Single Images to Pairs

Key Points

Discussion

Implication

Cite This Study

Keep reading

Keep reading