Select earthquake forecasting models demonstrate consistency with prospective decadal observations in California

Earthquake forecasting systems are now operationalized by agencies in several countries, providing situational awareness to at-risk communities. To ensure that forecasts follow the best science available, the underlying models require rigorous testing against prospective data and benchmarking. Between August 2007 and August 2018, 27 forecasting models, including the types used for operational earthquake forecasting, were prospectively operated as part of a multi-institutional project by the Collaboratory for the Study of Earthquake Predictability, generating 51,625 next-day M w ≥ 3.95 seismicity forecasts for California. Here, we comprehensively evaluate their predictive skills against 597 M w ≥ 3.95 earthquakes using community-vetted statistical methods. We show that approximately 95%, two-thirds, and 95% of the models produced forecasts consistent with the number, locations, and magnitudes of observed earthquakes, respectively. Individually, select versions of well-established, operational-type models demonstrated sustained consistency with prospective observations. We provide detailed recommendations, software tools, and benchmarking data to support the development of operational earthquake forecasting systems worldwide.