A Comparison of Five Different Scoring Functions for Confidence Tests¹

Several parts of the STEP Writing Test, Level 1, were administered to 14 different groups of from 19 to 52 high school students. In the testing situations, scores were computed using the following scoring functions: (a) probability assigned to the correct answer, (b) the logarithmic function, (c) the spherical function, (d) the Euclidean function, and (e) inferred choice. Reliabilities of the scores obtained by means of each scoring function were computed. Comparisons between the reliabilities showed that the simplest and most intuitive function, the probability assigned to the correct answer, produced the highest reliability in comparison with any of the other functions. The data suggest that in the absence of information about the scoring system, subjects assign their confidence in multiple‐choice responses on the basis of the intuitively simplest payoff model, and that reliability decreases as scoring functions generate item scores which are progressively discrepant from scores generated by the simplest model.