HCFNet: Hierarchical and complementary feature embedding for remote sensing change captioning

Remote sensing change captioning (RS-CC) aims to describe changes between bi-temporal remote sensing images in natural language. Existing methods commonly encode visual changes into a single fused difference feature, which may limit the utilization of change information with different extents during caption generation. In addition, conventional n -gram-based metrics are insensitive to mismatches in spatial scale and change extent. To address these issues, a Hierarchical and Complementary Feature Embedding Network (HCFNet) is proposed. Specifically, the Change-Cognitive Visual Parser (CCVP) extracts branch-specific change embeddings from the fused difference feature through parallel aggregation with different spatial preferences. Hierarchical Cognition Mining Loss (HCMLoss) utilizes the change intensity derived from the change mask to form the Hierarchical Cognitive Vector (HCoV). The Change-Cognitive Language Decoder (CCLD) further incorporates branch-specific HCoV into different decoding layers through multi-stage feature fusion. In addition, the Change-Sensitive Metric (CSM) complements traditional n -gram-based metrics with a change-scene penalty term, thereby improving sensitivity to spatial-scale and change-extent mismatches. Experiments are conducted on three benchmark datasets, LEVIR-CC, WHU-CDC, and SECOND-CC. The results demonstrate that the proposed HCFNet achieves competitive performance, validating the effectiveness of HCoV in remote sensing change captioning. The implementation code and vocabulary of CSM will be publicly available at https://github.com/Hope21maker/HCFNet .