Optimally Persistent Patterns in Time-Varying Fields

A technique is described for determining the set of patterns in a time-varying field whose corresponding time series remain correlated for the longest times. The basic idea is to obtain patterns that, when projected on a time-varying field, produce time series that optimize a measure of decorrelation time. The decorrelation time is measured by one of the integrals where τ is the time lag and ρτ is the correlation function of the time series. These integrals arise naturally in sampling theory and power spectra analysis. Moreover, these integrals define the maximum lead time beyond which linear prediction models lose all forecast skill. Thus, an optimally persistent pattern is interesting because it optimizes a quantity that is of fundamental and practical importance. An orthogonal set of time series that optimize these integrals can be obtained from the lagged covariance matrix of the dataset. The corresponding patterns, called optimal persistence patterns (OPPs), may provide a useful basis set for statistical prediction models, because they may remain correlated for much longer periods than individual empirical orthogonal functions (EOFs). The main shortcoming of OPPs is that they are sensitive to sampling errors. To reduce the sensitivity, the upper limit of integration and the basis set used to define the pattern need to be as small as possible, yet large enough to resolve the space–time structure of the pattern. Examples of OPPs are presented for the Lorenz model and the daily anomaly 500-hPa geopotential height fields. In the case of the Lorenz model, the technique is shown to be far superior at capturing persistent, oscillatory signals than other techniques. As for geopotential height, the technique reveals that the absolute longest decorrelation time, in the space spanned by the first few dozen EOFs, is 12–15 days. It is perhaps noteworthy that this time is virtually identical to the theoretical limit of atmospheric predictability determined in previous studies. This result suggests that the monthly anomaly in this state space, which is often used to study long-term climate variability, arises not from a perturbation that lasts for a month, but rather from a few “episodes” often lasting less than 2 weeks. Depending on the number of EOFs and on which measure of decorrelation time is considered, the leading OPP resembles the Arctic oscillation. The second OPP is associated with an apparent discontinuity around March 1977. The OPP that minimizes decorrelation time (the “trailing OPP”) is associated with synoptic eddies along storm tracks. The technique not only finds persistent signals in stationary data, but also finds trends, discontinuities, and other low-frequency signals in nonstationary data. Indeed, for datasets containing both a random component and a nonstationary component, maximizing decorrelation time is shown to be equivalent to maximizing the signal-to-noise ratio of low-frequency variations. The technique is especially attractive in this regard because it is very efficient and requires no preconceived notion about the form of the nonstationary signal.