MFVG: A Visual Grounding Network with Multi-scale Fusion

Visual grounding, as a crucial multimodal reasoning task, aims to locate target objects in images based on natural language queries. This task requires the model to perform multimodal fusion and reasoning effectively. Early methods often rely on complex and manually designed modules for multimodal fusion and reasoning. However, these methods are usually customized for certain specific scenarios, thus limiting the generalization ability of the model. Recent works achieve visual grounding through the attention mechanism, which can capture the alignment relationship between vision and language, but ignore the importance of different scale features for multimodal reasoning. This paper proposes MFVG, a concise and effective visual grounding framework based on multiscale fusion guided by texts, which learns visual features with discriminative semantics through text queries. Specifically, MFVG allows the contextual semantic information of vision and language to interact fully and fuses features at different scales guided by text queries to capture richer detail features and semantic information, thereby enhancing the representational ability of the model and achieving better visual grounding. We conducted extensive experiments on five widely used benchmarks. The experiment results show that our proposed MFVG is superior to or comparable with the state-of-the-art methods.