Applying the Online Scoring Network (Osn) to Advanced Placement Program® (Ap®) Tests

This project explored the feasibility of using the ETS Online Scoring Network (OSN) to score selected Advanced Placement Program ® (AP ® ) tests. In particular, the quality of scores obtained from traditional large‐scale readings was compared with the quality of scores obtained from OSN scoring. (For the former, readers convene in a central location to evaluate student responses in a paper‐based format; for the latter, they work alone from remote locations to evaluate responses that are transmitted via the Internet.) The study also obtained readers' reactions to OSN training and to OSN capabilities for monitoring reader performance online. The results, based on more than 11,000 test takers who had taken either the AP English Language and Composition exam or the AP Calculus AB exam, revealed little if any difference between scores assigned at traditional central readings and those assigned when readers used OSN. The lack of differences applied to (a) score level and variability, (b) interreader agreement, and (c) passing rates on the two AP exams that were investigated. Study participants did, however, suggest several areas in which improvements could be made to OSN procedures.