MambaWDC: Efficient Weather Data Compression via Selective State Space Model

The storage and transmission of large-scale meteorological data have become a significant bottleneck hindering the in-depth development of meteorological research. This study proposes a novel meteorological data compression framework characterized by innovative thinking. It embeds the Mamba structure during the feature extraction stage and integrates a spatio-temporal channel attention mechanism to construct an entropy coding mechanism that facilitates the sharing of high-dimensional meteorological data channel information. The framework achieves a compression rate of 300x and, more importantly, preserves the essential spatio-temporal features of meteorological data. Validation on the ERA5 dataset demonstrates that the proposed method surpasses existing techniques in terms of both data compression efficiency and reconstruction accuracy. The accuracy of meteorological analyses based on compressed data is comparable to that of the original data, offering a highly practical solution to the challenges of storing meteorological big data. Our source code are publicly available online at https://github.com/cike0cop/MambaComp.