Selecting significant effects in factorial designs: Lenth’s method versus the Box-Meyer approach

The Lenth method is conceptually simple and probably the most common approach to analyzing the significance of the effects in factorial designs. Here, we compare it with a Bayesian approach proposed by Box and Meyer and which does not appear in the usual software packages. The comparison is made by simulating the results of 4, 8 and 16 run designs in a set of scenarios that mirror practical situations and analyzing the results provided by both methods. Although the results depend on the number of runs and the scenario considered, the use of the Box and Meyer method generally produces better results.