Quantifying Certainty in Design Decisions: Examining AHP

Quantification of stochastic uncertainty through confidence intervals when probability distributions are known is well-understood. There is considerable uncertainty in design decision support, however, for which probability distributions are unknown. The confidence interval formulation does not apply to these situations. The Analytic Hierarchy Process, or AHP, is an example of a tool with wide-spread industry application but questionable mathematical foundations. It is recognized by responsible practitioners that AHP should not be used as an optimization tool, but as a means of clarifying group attitudes. This raises the question, how differently must two alternatives be ranked by AHP to instill confidence that one is truly better than the other? In practice, this question is always answered using intuition. This paper examines and proposes the quantification of sources of uncertainty in the Analytic Hierarchy Process, and offers an answer to the question posed above. This study is the first step of a larger effort to quantify uncertainties in decision support in engineering design.