A clearer view of systematic errors in model development: two practical approaches using perturbed parameter ensembles

Abstract Models of weather and climate are continuously being developed to improve their reliability and, ultimately, to help users make more informed decisions. But they are often affected by inadequacies in the physical approximations they use, which can lead to ‘systematic’ errors. Exposing, understanding, and resolving these errors is therefore a key aim of model development. One technique that can readily expose systematic errors is the use of perturbed parameter ensembles (PPEs). Here, we show how PPEs can be used to effectively track the impacts of updates to the model’s structure, which are required to fix these errors. We demonstrate this using two PPEs based on recent configurations of the UK Met Office’s climate model (HadGEM3-GA7.05 and HadGEM3-GA8). We show there are systematic errors in cloud radiative effects in both PPEs, but also systematic improvements in the more recent HadGEM3-GA8 PPE. Further, we discuss how PPEs can provide a clear view of updates to the model—one which is not affected by model parameter tuning, which can mask the magnitude of some systematic errors. We also propose more practical, computationally cheaper, alternatives for use during model development: single variants using the mode of the prior parameter distributions for each PPE. We show that these ‘modal’ variants provide a better representation of the typical changes between the two PPEs than the ‘tuned’ variants, and we suggest they would be a valuable tool for evaluating the systematic errors that need to be fixed during model development.