Non-Dimensional Measures of Climate Model Performance

The characteristics of several non-dimensional measures of skill, which can be readily used to quantify the accuracy of simulated climatologica l fields, are examined. The correlation coefficient, the ‘SITES1’ measure, and two forms of Willmott's ‘index of agreement’ are compared with Mielke's measure of agreement, ρ=1−δ/μ, where δ is either mean square or mean absolute error, and μ is the expected value of δ if the simulated values are distributed randomly over the global grid-points. It is shown that, after certain transformations, the measures converge similarly to unity, for small errors. Fields simulated by the CSIRO9 general circulation model are used to illustrate the behaviour of the measures. Although all the measures can be useful, it is shown that the transformation M of the mean square ρ, where M= arcsin(ρ)/π, is especially practical. Three examples of its use are given. In comparing the skill of several climate models in simulating the global distribution of seasonal mean sea-level pressure, M ranged from 0·02 to 0·75. In comparing the skill of CSIRO9 in simulating various climatological quantities, M values ranged from 0·23 for cloud cover to 0·85 for surface air temperature. In comparing present and doubled CO2; climates simulated by CSIRO9, the quantity with the largest change, relative to its spatial variation, is the water vapour column (M=0·67) and that with the smallest change is sea-level pressure (M=0·90).