Qualitative Analysis of Oral Reading Errors: The Inter-Judge Reliability of Scores

A DETAILED SET OF INSTRUCTIONS for judging the quality of oral reading errors was prepared in an attempt to obtain adequate reliability from one judge to another-between judges of scores. The instructions included definitions of the oral errors and rules for scoring graphic similarity, corrections, repetitions, and contextual appropriateness. Five female college graduates scored 45 samples of oral reading obtained from second grade, average readers who read two 233-word stories. The readability levels of the stories were estimated to be second and third grade. The training of judges required 5 days, 6 hours per day. The scoring required an average of 2 hours per subject per judge. Each subject's scores were means of the scores obtained by the 5 judges. Alpha reliabilities of scores for total errors, meaning loss errors, repetitions, and proportions of graphically similar errors and errors self corrected by the child in the combined stories ranged from .97 to .99. Alpha reliabilities of scores for contextual appropriateness ranged from .84 to .94.