Hierarchical Testing of a Hybrid Machine Learning-Physics Global Atmosphere Model

Machine learning (ML)-based models have demonstrated high skills and computational efficiency, often outperforming conventional physics-based models in weather and subseasonal predictions. While prior studies have assessed their fidelity in capturing synoptic-scale atmospheric dynamics, their performance across timescales and under out-of-distribution forcing, such as +3K or +4K uniform-warming forcings, remains unclear, which is crucial in establishing model's reliability for Earth science. Here, we design three idealized experiments targeting synoptic-scale phenomena, interannual variability, and out-of-distribution uniform-warming forcings. We evaluate the Neural General Circulation Model (NeuralGCM), a hybrid model integrating dynamical core with ML-based component, against observations and physics-based Earth system models (ESMs). At the synoptic scale, NeuralGCM captures the evolution and propagation of extratropical cyclones with performance comparable to ESMs. At the interannual scale, NeuralGCM successfully reproduces teleconnection patterns and nonlinearity over the Pacific/North America regions when forced by El Niño-Southern Oscillation (ENSO)-like sea surface temperature (SST) anomalies. Under out-of-distribution uniform-warming forcings, NeuralGCM simulates reasonable responses in global-average temperature and precipitation and reproduces large-scale tropospheric circulation features similar to those in ESMs. Notable weaknesses include overestimating the tracks and spatial extent of extratropical cyclone, biases in the teleconnected wave train triggered by tropical SST anomalies, and differences in upper-level warming and stratospheric circulation response to SST warming compared to physics-based ESMs. The causes of these weaknesses were explored. Despite the noted weaknesses, NeuralGCM reproduces responses across experiments reasonably and performs comparably to observations and ESMs. By integrating a dynamical core with ML, NeuralGCM shows potential as a step toward developing ML-based ESMs.