Mining Hierarchical Information of CNNs for Scene Classification of VHR Remote Sensing Images

Scene classification of very high resolution (VHR) images is an active research subject in remote sensing community, and it has provided data or decision supports for many practical applications. Although existing CNN-based methods have achieved good classification results, they have not fully exploited rich potential information contained in pre-trained models. In this paper, a novel framework termed hierarchical features fusion of convolutional neural network (HFFCNN) is developed for scene classification of VHR images. On the whole, the HFFCNN covers two parallel modules to severally process convolutional features and fully connected (FC) features. At the first module, an adaptive spatial-wise attention based multi-scale nonlinear bag-of-visual-words (ASA-MNBoVW) model is designed to encoding convolutional feature maps, and the responses of discriminative regions are highlighted without introducing any additional parameters. For the second module, a weighted image pyramid structure is adopted to reveal geometric information and spatial layouts by aggregating local image patch-based FC features. Finally, these hierarchical features are combined for mutually complementing, and a linear classifier is adopted to predict semantic labels. Experimental results organized on two challenging data sets prove that the developed HFFCNN approach obtains more dramatic performance of scene classification than some state-of-the-art methods in terms of OAs.