The Last Mile in Remote Sensing Poverty Prediction

A prominent approach in remote sensing poverty prediction uses deep learning satellite imagery with daytime and nighttime light (NTL). While results yield acceptable average predictions, certain areas consistently perform poorly. For high-stakes aid allocation decisions, however, achieving near-perfect predictions across all regions is crucial. To address this, we apply explainability techniques to evaluate vision models, enhance predictions with multimodal architectures, and investigate challenging cases. Our diagnostic analysis shows that fine-tuning with NTL tends to fail in areas where NTL poorly reflects wealth. Integrated Gradients and Guided Grad-CAM techniques also reveal that VGG, ResNet, and ViT architectures tend to focus on brightly lit features, such as roads and buildings. To complement vision models, a multimodal model integrates non-visual features. Metrics such as X (formerly known as Twitter) activity, distance from residential roads, and internet speed are especially predictive. Among 10 tested architectures, the Stack Ensemble Late Stage model, combining pre-trained weights, multimodal data, and NTL transfer learning, achieves the best performance with high accuracy and low error variance. Qualitative analysis of difficult-to-predict regions highlights the challenges of modeling wealth due to inherent aleatoric uncertainty. Two core issues, spatial inequality in small areas and the presence of rare features, underscore the need for principled cluster segmentation.