Incorporating Prior Information in Optimal Design for Model Selection

An important use of experimental designs is in screening, in which experimenters seek to identify significant effects (both main effects and potentially interactions) from a large set of candidate effects. This article goes further than identification of effects, introducing a design criterion that seeks to maximize the ability to discriminate between models. Motivated by the work of Meyer, Steinberg, and Box, the Bayesian criterion is based on the Hellinger distance between predictive distributions under competing models. A bound for the criterion is obtained, greatly improving interpretability. The set of all possible models to compare is huge, and not all models are equally plausible. This challenge is addressed through prior distributions on the space of models that indicate preference for intuitively appealing models, such as those with few effects, more low-order than high-order effects, and inheritance structure between active main effects and interactions. Techniques for evaluating the criterion and searching for optimal designs are presented. The effectiveness of the criterion is illustrated with a number of examples that consider regular and nonregular designs, robust designs, and scenarios with partial prior knowledge of which effects are significant.