Coupled Local and Global Semantic Alignment for Image-Text Matching

The regional features in the image tend to focus more on the foreground information of the image, and the global features in the image tend to focus more on the important background information in the image. How to effectively integrate the foreground information and background information in the image have become an urgent problem to be solved. To address the above issus, we propose a novel Coupled Local and Global Semantic Alignment network. First we propose global image features of images to preserve important background information in images. Second we extract regional image features to preserve fine-grained semantic information in images. Then we utilize the GRU network to extract features for each word in the text. We align image features and text features globally and locally to obtain global similarity and local similarity. We assign different similarity weights to the similarity matrix to aggregate the similarity matrix. Extensive experimental results demonstrate the effectiveness of our proposed method.