Flow-Dependent Evaluation and Training of Random Forest based Probabilistic Forecasts of Severe Weather Hazards

There has been an increasing interest in recent years in the use of machine learning (ML) algorithms such as random forests (RF) in the context of severe weather prediction. However, there is a need to better understand the impacts of large-scale flow patterns on RF model performance to further improve their use of multi-scale information. This work leverages real-time forecasts from convection-allowing FV3-based ensemble forecasts produced by the University of Oklahoma Multi-scale data Assimilation and Predictability Lab during the Hazardous Weather Testbed (HWT) Spring Forecasting Experiments. Results show that by using an RF trained on all forecast cases, forecast cases that have relatively high importance for certain predictors have different discernible flow patterns compared to low importance forecast cases for the same predictors through composite differences of different environmental variables. Maximum updraft helicity storm attribute predictors were associated with strong synoptic ascent, u 500 had compact shortwaves within northwesterly flow, and MUCAPE was associated with forcing displaced north of the region of severe weather. RF models trained on forecast cases with similar domain-averaged CAPE/shear were statistically more skillful than the baseline model trained irrespective of CAPE/shear patterns in forecasting the occurrence of severe weather. However, selecting training forecast cases based on spatial patterns or principal components of CAPE/shear using EOF analysis did not further improve the RF ability to forecast severe weather compared to the baseline model. The benefits of training on forecast cases based on domain-averaged CAPE/shear were maintained, and some of the benefits of training based on spatial patterns of CAPE/shear were maintained as the sample size increased.