An Operations-Based Evaluation of Tropical Cyclone Track and Intensity Forecasts from Artificial Intelligence Weather Prediction Models

Abstract In just the past few years multiple data-driven Artificial Intelligence Weather Prediction (AIWP) models have been developed. NHC verification procedures are used to evaluate seven-day track and intensity predictions for northern hemisphere TCs from May-November 2023. Four open-source AIWP models initialized four times per day with GFS initial conditions are considered (FourCastNetv1, FourCastNetv2-small, GraphCast-operational and Pangu-Weather). The AIWP track forecast errors and detection rates are comparable to those from the best-performing operational forecast models. However, the AIWP intensity forecast errors are larger than those of even the simplest intensity forecasts based on climatology and persistence. The AIWP models almost always reduce the TC intensity resulting in a substantial low bias. An idealized Rankine vortex was used to illustrate how minimizing RMS errors in the AIWP models contributes to this bias. The contribution of the AIWP models to the NHC model consensus was also evaluated. For track, adding the AIWP models improves the consensus by up to 11% at the longer time periods. This represents more than a five-year gain in accuracy based on the historical track forecast improvement of about 2% per year. In contrast, the AIWP models did not improve the intensity consensus. A preliminary evaluation of the impact of the AIWP model initial condition was performed by comparing forecasts initialized with GFS and ECMWF-IFS initial conditions for northern hemisphere cases in 2022-2023. With the ECMWF-IFS initial conditions the AIWP track forecasts significantly improved, but there was little improvement in the intensity forecasts.