Frequency-Domain Enhanced Cross-modal Interaction Mechanism for Joint Video Moment Retrieval and Highlight Detection

This is the authors' abstract. We don't add key points for this paper.

The joint video moment retrieval and highlight detection aims to locate moments and highlight clips relevant to textual queries. The main challenges lies in the fusion of multi-modal data and understanding semantic information. Current research for addressing this issue primarily focus on capturing information from different modalities in temporal order. Due to the heterogeneity of different modalities, it is difficult to capture all complementary information in temporal domain. Therefore, we propose a frequency-domain enhanced cross-modal interaction mechanism (FCIM) to capture complementary information from different modalities across all frequency components, addressing the limitations in the temporal domain. Additionally, we design a lightweight inference model for the joint (LIMJ) task. Extensive experiments on three datasets demonstrate the effectiveness and generality of our method.