Training Effectiveness Measures and Scoring Schemes: A Comparison

This paper examines a new criterion measure, comprised of scripted situations, for training effectiveness and compares two scoring schemes–one based on expert judgments and one based on empirical weights. It was believed that a measure based on job‐related scenarios would be more sensitive to detecting change due to training than would more traditional training criteria, such as behavior and performance rating scales. Results of the study support this notion. For a sample of 67 teachers, significant training effects were found only for the new training effectiveness measure; there were no significant differences found for traditional behavior and performance measures. The comparison of expert versus empirical scoring indicated that empirical scoring produced significant results, while scoring based on expert judgment did not. Implications of these findings are discussed.