On the validation of models of forest CO2 exchange using eddy covariance data: some perils and pitfalls

With the widespread application of eddy covariance technology, long-term records of hourly ecosystem mass and energy exchange are becoming available for forests around the world. These data sets hold great promise for testing and validation of models of forest function. However, model validation is not a straightforward task. The goals of this paper were to: (1) review some of the problems inherent in model validation; and (2) survey the tools available to modelers to improve validation procedures, with particular reference to eddy covariance data. A simple set of models applied to a data set of ecosystem CO2 exchange is used to illustrate our points. The major problems discussed are equifinality, insensitivity and uncertainty. Equifinality is the problem that different models, or different parameterizations of the same model, may yield similar results, making it difficult to distinguish which is correct. Insensitivity arises because the major sources of variation in eddy covariance data are the annual and diurnal cycles, which are represented by even the most basic models, and the size of the response to these cycles can mask effects of other driving variables. Uncertainty arises from three main sources: parameters, model structure and data, each of which is discussed in turn. Uncertainty is a particular issue with eddy covariance data because of the lack of replicated measurements and the potential for unquantified systematic errors such as flux loss due to advection. We surveyed several tools that improve model validation, including sensitivity analysis, uncertainty analysis, residual analysis and model comparison. Illustrative examples are used to demonstrate the use of each tool. We show that simplistic comparisons of model outputs with eddy covariance data are problematic, but use of these tools can greatly improve our confidence in model predictions.