Self-Supervised Learning for Spaceborne SAR and Multispectral Image Representation With Few-Shot Local Climate Zone Classification

This is the authors' abstract. We don't add key points for this paper.

Deep learning has demonstrated significant potential in remote sensing scene classification; nevertheless, its efficacy is frequently hindered by the requirement for extensive labeled datasets and its restricted generalisation in practical applications. We developed a Self Supervised multimodal representation Learning (MMRL) framework for Local Climate Zone Classification (LCZC) to tackle these problems. Our approach utilises novel encoder architecture that derives representations exclusively from synthetic aperture radar (SAR) and multispectral (MS) data. To enhance the learning process, we implemented multimode loss and consistency loss, enhancing the model ability to prioritize the most meaningful features. The encoder, trained on the unlabelled So2Sat-LCZ42 dataset, acquires robust and transferable representations, further refined for a few-shot LCZ classification task utilising a restricted number of labelled samples. This approach allows the model to efficiently utilize extensive unlabeled data, leading to enhanced performance in subsequent classification tasks. Experimental findings on the So2Sat-LCZ42 benchmark validate that our self supervised learning (SSL) based approach achieved better accuracy than the current state-of-the-art(SOTA) SSL models, illustrating its efficacy for label-efficient LCZC.