The Reliability of Test Candidates and the Implications for One‐shot Testing

The results of administering alternative forms of applied tests indicate that the reliability of candidates’ performance deserves increased attention in relation to both the interpretation of examination results and test constructors’ reliance on internal reliability measures. In an investigation of candidate reliability, subjects drawn from youth training schemes (in further education and industry) each completed two exercises drawn from a range of alternative forms and involving location of information, various arithmetic calculations and invoice completion. There was found to be very little relationship between candidates’ two scores, even though the tests were characterised by high internal reliability. These findings have serious implications for a system in which most examination results reflect performance on a single occasion. Moreover, while the guidelines for the National Curriculum and for National Vocational Qualifications formally endorse assessment over a period of time, their long lists of separate attainment targets actually encourage ‘one‐off’ testing by teachers faced with long lists of assessments to complete. The discussion underlines the risks attendant on such a trend.