MM-RNN: A Multimodal RNN for Precipitation Nowcasting

Precipitation nowcasting, the high-resolution forecasting of precipitation in a short term, is essential in various applications in the real world. Previous deep learning methods use huge samples to learn potential laws, and the learning process lacks regularity, making it difficult to model the complex nonlinear precipitation phenomenon. Inspired by traditional numerical weather prediction models, we propose the MultiModal RNN (MM-RNN), which introduces knowledge of elements to guide precipitation prediction. This constraint forces the movement of precipitation to follow the underlying atmospheric motion laws. MM-RNN not only can provide accurate precipitation nowcasting but other meteorological elements predictions. Besides, it has high flexibility and is compatible with multiple RNN models, such as ConvLSTM, PredRNN, MIM, MotionRNN, etc. We conduct experiments on two multimodal datasets (MeteoNet and RAIN-F) and the results indicate that MM-RNN is superior to common RNN (MultiScale RNN, MS-RNN) using a single radar modality. For the MeteoNet, compared to MS-MotionRNN, the CSI (R ⩾ 10) of MM-MotionRNN increases by 23.4%, and the MSE of MM-MotionRNN decreases by 6.7%. For the RAIN-F, compared to MS-MIM, the HSS (R ⩾ 5) of MM-MIM increases by 209.4%, and the B-MSE of MM-MIM decreases by 4.6%.