Self-Supervised Joint Representation Learning for Urban Land-Use Classification With Multisource Geographic Data

Urban land-use patterns furnish the functional structures of cities, playing an essential role in urban planning and management. Recently, advancements in remote sensing and Internet technologies have made high-resolution remote sensing (HRS) images and social sensing data more widely available. These multisource geographic data contain comprehensive urban scene information, providing new opportunities for urban land-use classification. However, as the social sensing data are sparsely distributed, learning representations of parcels for land-use classification with limited multisource sample pairs remains challenging. In this article, a self-supervised joint representation learning (SJRL) framework for urban land-use classification with multisource geographic data is proposed. Specifically, a semantic-aware self-supervised representation learning approach is designed to mine visual information from the HRS images. This approach introduces a semantic-aware sampling strategy to identify regions with significant visual information, thereby enhancing the efficiency of the representation learning. For the points of interest (POIs), which are a type of social sensing data, a context-aware self-supervised representation learning approach is proposed to obtain a pretrained POI model suitable for land-use classification. To capture the correlation of multisource data, a cross-modal alignment (CMA) module and a modality-enhanced module (MEM) are designed to align the common information across modalities and adaptively learn the complementary information between multisource data. The CMA and MEM are then embedded into the land-use classification model for accurate classification. Experiments conducted on multisource samples from 34 Chinese provincial cities and the urban regions of Beijing, Shanghai, Wuhan, and Chengdu in China verified the effectiveness and generalizability of the proposed SJRL framework.