Huge Ensembles of Weather Extremes using the Fourier Forecasting Neural Network

Studying low-likelihood high-impact extreme weather and climate events in a warming world requires massiveensembles to capture long tails of multi-variate distributions. In combination, it is simply impossible to generatemassive ensembles, of say 10,000 members, using traditional numerical simulations of climate models at highresolution. We describe how to bring the power of machine learning (ML) to replace traditional numericalsimulations for short week-long hindcasts of massive ensembles, where ML has proven to be successful in terms ofaccuracy and fidelity, at five orders-of-magnitude lower computational cost than numerical methods. Becausethe ensembles are reproducible to machine precision, ML also provides a data compression mechanism toavoid storing the data produced from massive ensembles. The machine learning algorithm FourCastNet (FCN) isbased on Fourier Neural Operators and Transformers, proven to be efficient and powerful in modeling a widerange of chaotic dynamical systems, including turbulent flows and atmospheric dynamics. FCN has already beenproven to be highly scalable on GPU-based HPC systems.  We discuss our progress using statistics metrics for extremes adopted from operational NWP centers to showthat FCN is sufficiently accurate as an emulator of these phenomena. We also show how to construct hugeensembles through a combination of perturbed-parameter techniques and a variant of bred vectors to generate alarge suite of initial conditions that maximize growth rates of ensemble spread. We demonstrate that theseensembles exhibit a ratio of ensemble spread relative to RMSE that is nearly identical to one, a key metric ofsuccessful near-term NWP systems. We conclude by applying FCN to severe heat waves in the recent climaterecord.