Minimum relative entropy: Forward probabilistic modeling

The pioneering work of Jaynes in Bayesian/maximum entropy methods has been successfully explored in many disciplines. The principle of maximum entropy (PME) is a powerful and versatile tool of inferring a probability distribution from constraints that do not completely characterize that distribution. Minimum relative entropy (MRE) is a method which has all the important attributes of the maximum entropy approach with the advantage that prior information may be easily included. In this paper we use MRE to determine the prior probability density function (pdf) of a set of model parameters based on limited information. The resulting pdf is used in Monte Carlo simulations to provide expected values in field variables such as concentration, and confidence limits. We compare the probabilistic results from a traditional advection‐dispersion (ADE) model based on volumetric averaging concepts with that of a model based on the assumption that the hydraulic conductivity is a stationary stochastic process. The results suggest that although Naff's (1990) model satisfies the observed data to a better degree than ADE model, the upper and lower confidence bands about the mean value are larger than the ADE results. This result we attribute to the fact that Naff's (1990) model simply contains more parameters, each of which is unknown and has to be estimated. There is no statistical difference between the expected values of second‐spatial moments for the two models. The examples presented in this paper illustrate problems associated with assigning Gaussian pdfs as priors in a probabilistic model. First, such an assumption for the input parameters actually injects more information into the problem than may actually exist, whether consciously or unconsciously. This fact is borne out by comparison with minimum relative entropy theory. Second, the output parameters as suggested from the Monte Carlo analysis cannot be assumed to be Gaussian distributed even when the prior pdf is Gaussian in form. In a practical setting, the significance of this result and the approximation of Gaussian form would depend on the toxicity and environmental standards that apply to the problem.