Measuring the Visual Hallucination in ChatGPT on Visually Deceptive Images

This is the authors' abstract. We don't add key points for this paper.

The evaluation of visual hallucinations in multimodal AI models is novel and significant because it addresses a critical gap in understanding how AI systems interpret deceptive visual inputs. The study systematically assessed ChatGPT's performance on a synthetic dataset of visually deceptive and non-deceptive images, employing both quantitative and qualitative analysis. Results revealed that while ChatGPT achieved high accuracy in standard visual recognition tasks, its performance diminished when faced with deceptive images, highlighting areas for further improvement. The analysis provided insights into the model's underlying mechanisms, such as its extensive pretraining and sophisticated multimodal integration capabilities, which contribute to its robustness against visual deceptions. The study's findings have important implications for the development of more reliable and robust AI technologies, offering a benchmark for future evaluations and practical guidelines for enhancing AI systems.