Reconciling theory with observations: elements of a diagnostic approach to model evaluation

Abstract This paper discusses the need for a well‐considered approach to reconciling environmental theory with observations that has clear and compelling diagnostic power. This need is well recognized by the scientific community in the context of the ‘Predictions in Ungaged Basins’ initiative and the National Science Foundation sponsored ‘Environmental Observatories’ initiative, among others. It is suggested that many current strategies for confronting environmental process models with observational data are inadequate in the face of the highly complex and high order models becoming central to modern environmental science, and steps are proposed towards the development of a robust and powerful ‘Theory of Evaluation’. This paper presents the concept of a diagnostic evaluation approach rooted in information theory and employing the notion of signature indices that measure theoretically relevant system process behaviours. The signature‐based approach addresses the issue of degree of system complexity resolvable by a model. Further, it can be placed in the context of Bayesian inference to facilitate uncertainty analysis, and can be readily applied to the problem of process evaluation leading to improved predictions in ungaged basins. Copyright © 2008 John Wiley & Sons, Ltd.