Test Speededness Under Number‐right Scoring: An Analysis of the Test of English as a Foreign Language

ABSTRACT This investigation aimed to develop a new methodology for assessing the speededness of right‐scored tests such as the Test of English as a Foreign Language (TOEFL). A new methodology is needed because traditional procedures of assessing speededness assume that the test is scored under formula‐scoring instructions. The approach to the problem used in this study capitalizes on the fact that Item Response Theory is used in equating TOEFL. Two indices were investigated and validated against artificial data and real data in a way that was consistent with the ETS criteria for declaring forms speeded. Both indices were able to identify data that contained the speed component. The applications of these indices to several forms suggest that speed does not play a significant role in the TOEFL as a whole. However, Section 2 appeared to be affected by speed in administrations where pretest items are included. This may have been due to the slightly shorter time per item allowed when pretest items are included. It was recommended that one of the indices studied be used on an operational basis for a trial period and that the time limit for Section 2 be increased when pretest items are present or that the number of items be reduced to achieve the same effect.