This paper presents Modular Arabic Visual Question Answering (VQA), a novel system that leverages a language-mediated framework to perform open-ended VQA task in Arabic. Unlike traditional vision-language models that rely on multimodal pretraining and dense visual embeddings, our approach represents images entirely through natural language. The system integrates multilingual and Arabic image captioning models, namely AraBERT32-Flickr8k, Violet, and GPT-4o, to generate context-rich image descriptions, which are then processed by a frozen large language model (LLM), Gemini 1.5 Flash, for answer generation. Experimental results on two large-scale Arabic-translated VQA datasets, VQAv2-ar and OKVQA-ar, demonstrate that replacing visual embeddings with text-based captions significantly enhances performance. The best configuration, combining GPT-4o captions with Gemini 1.5 Flash, achieved improvements of up to 9.5% in Fuzz Accuracy compared to visual-token-based baseline. This study provides the first comprehensive evaluation of language-mediated Arabic VQA and highlights the potential of modular, text-driven designs for multilingual visual reasoning. Future directions include developing question-conditioned caption generation to further strengthen contextual alignment between visual and linguistic cues. The complete implementation of the Modular Arabic VQA system is publicly released for research purposes 1 .
Alshalawi et al. (Thu,) studied this question.