A Study of the Performance of Subjective Probability Assessors

Despite the key role that subjective probabilities play in decisions made under conditions of uncertainty, little is known about the ability of probability assessors in developing these estimates. A literature survey is followed by a review of results from a continuing series of experiments designed to investigate the external accuracy of subjectively assessed probability distributions. Initial findings confirm that probability assessments provided by untrained assessors are of questionable value in predicting the distribution of actual outcomes of uncertain events. Particular difficulty is encountered when subjects attempt to quantify the extremes of their subjective distributions. The impact of extended assessor training and hypotheses regarding the effects of variation in the assessor's information level and the complexity of the assessment task are explored. Implications for applied decision making are drawn, and directions for future investigations are suggested.