Seeing Through Clouds in Satellite Images

This article presents a neural network-based solution to recover pixels occluded by clouds in satellite images. We leverage radio frequency (RF) signals in the ultrahigh-/superhigh-frequency band that penetrates clouds to help reconstruct the occluded regions in multispectral images. We introduce the first multimodal multitemporal method for cloud removal. Our model uses publicly available satellite observations and produces daily cloud-free images. Experimental results show that our system outperforms several baselines on multiple metrics. We also demonstrate use cases of our system in digital agriculture, flood monitoring, and wildfire detection.