Classification of pixel-level fused hyperspectral and lidar data using deep convolutional neural networks

We investigate classification from pixel-level fusion of Hyperspectral (HSI) and Light Detection and Ranging (LiDAR) data using convolutional neural networks (CNN). HSI and LiDAR imaging are complementary modalities increasingly used together for geospatial data collection in remote sensing. HSI data is used to glean information about material composition and LiDAR data provides information about the geometry of objects in the scene. Two key questions relative to classification performance are addressed: the effect of merging multi-modal data and the effect of uncertainty in the CNN training data. Two recent co-registered HSI and LiDAR datasets are used here to characterize performance. One was collected, over Houston TX, by the University of Houston National Center for Airborne Laser Mapping with NSF sponsorship, and the other was collected, over Gulfport MS, by Universities of Florida and Missouri with NGA sponsorship.