Statistical comparison of spatial fields in model validation, perturbation, and predictability experiments

The comparison of spatial fields of meteorological variables is an essential component of model validation studies and is central in assessing the significance of any change between a perturbed and control run of a general circulation model. Comparisons may be made of statistics which define the time‐mean state, the temporal variability about this state, and/or spatial variability. Comparisons may also be made of the two time‐mean spatial patterns, or of the temporal evolutions of spatial patterns. We consider here a suite of univariate and multivariate statistics which may be used to make these comparisons. Some of these statistics have been used previously, while others are either new or have not previously been used in the present context. The use of these statistics, their differences and similarities, and their relative performances are illustrated by considering mean sea level pressure changes between the decades 1951–1960 and 1971–1980 over an area covering North America, the North Atlantic Ocean, and Europe. Significance levels are assessed using the pool‐permutation procedure of Preisendorfer and Barnett (1983) (henceforth P+B). This overcomes problems arising from nonideal behavior of the data (particularly spatial autocorrelation), unknown sampling distributions, and multiplicity in the case of univariate statistics. A subset of statistics is identified as most useful. For tests of differences in means these are the grid point by grid point t ‐test, a test comparing the overall means, and P+B's SITES statistic. For tests of differences in temporal variability they are the grid point by grid point F ‐test, and SPRET1 (the ratio of the spatial means of the time variances). SPRET1 is a modification of P+B's SPRED statistic designed to identify the direction of any variance difference. As a test of spatial variability differences, we identify SPREX1 (the ratio of the time means of the spatial variances), and for comparing spatial patterns the best statistic is the (spatial) correlation coefficient between the time‐mean fields. For comparing the temporal evolution of spatial patterns, we recommend using the time‐mean anomaly field correlation which is a more easily interpreted equivalent to P+B's SHAPE statistic.