This study aims to develop a Computerized Adaptive Testing (CAT) system for efficiently and accurately diagnosing the general English proficiency of college students, and to empirically validate its effectiveness. To this end, an item bank was constructed by applying Item Response Theory (IRT) analysis to 100 TOEIC practice items (50 listening, 50 reading). The CAT system was subsequently designed with clearly defined components, allowing flexible administration in both fixed-and variable-length formats. The effectiveness of the system was then evaluated with 60 college students enrolled in two sections of a general English course at H University. The results indicated that, in the listening domain, the CAT group demonstrated significantly greater score gains compared to the non-CAT group (interaction effect). In the reading domain, both groups showed score improvement (learning effect); however, only the score gains of the CAT group reached statistical significance (simple effects). These findings provide empirically support for the utility of an IRT-based CAT system as a diagnostic tool capable of achieving both measurement precision and administrative efficiency in assessing general English proficiency. Furthermore, the results suggest that repeated use of the system as a supplementary learning tool may contribute positively to the development of English language ability.
Park et al. (Thu,) studied this question.