With the diversification of information retrieval methods, news retrieval tasks have gradually evolved towards multimodal retrieval. Existing methods often encounter issues such as inaccurate alignment and unstable feature matching when handling cross-modal data like text and images, limiting retrieval performance. To address this, this paper proposes an innovative multimodal news retrieval method by introducing the Learnable Alignment Module (LAM), which establishes a learnable alignment relationship between text and images to improve the accuracy and stability of cross-modal retrieval. Specifically, the LAM, through trainable label embeddings (TLEs), enables the text encoder to dynamically adjust category information during training, thereby enhancing the alignment capability of text and images in the shared embedding space. Additionally, we propose three key alignment strategies: logits calibration, parameter consistency, and semantic feature matching, to further optimize the model’s multimodal learning ability. Extensive experiments conducted on four public datasets—Visual News, MMED, N24News, and EDIS—demonstrate that the proposed method outperforms existing state-of-the-art approaches in both text and image retrieval tasks. Notably, the method achieves significant improvements in low-recall scenarios (R@1): for text retrieval, R@1 reaches 47.34, 44.94, 16.47, and 19.23, respectively; for image retrieval, R@1 achieves 40.30, 38.49, 9.86, and 17.95, validating the effectiveness and robustness of the proposed method in multimodal news retrieval.
Song et al. (Sun,) studied this question.