Model validation based on value-of-information theory

The modeling and simulation community has devoted considerable attention to the question of model validity as a condition for the use of a model in an engineering decision-making process. Their work has focused on the concept of “accuracy”, loosely defined as the difference between a model-computed result and a real-world result. The objective of this paper is to introduce an alternative approach, based on classical decision theory, that focuses on the value of the information that a model provides to the decision-making process. This is a significant departure from the current approach to model validation, and it derives from the preference, “I want the best outcome that I can get”. Use is made of an example case that results in a paradox to illustrate weaknesses in the accuracy-focused approach. Instead of advocating the use of a model based on its accuracy, this work advocates using a model if it adds value to the overall application, thus relating validation directly to system performance. The approach fills significant gaps in the current theory, notably providing a clearly defined validity metric and a mathematically rigorous rationale for the use of this metric.