Post hoc score category collapsing for L2 pronunciation research

Second language (L2) pronunciation research has measured speech comprehensibility by asking listeners to assess L2 learners’ speaking performance with rating scales. While some studies have provided validity evidence for these rating scales, few studies have examined the extent to which those scales effectively distinguish among L2 speakers. To fill this gap, the present study examines the 9-point scale used in Saito et al. (2020: Annual Review of Applied Linguistics, 40 , 9–25.) and the 100-point scale in Huensch and Nagle (2023: Studies in Second Language Acquisition, 45 (2), 571–585.) from a Rasch measurement perspective and showcases post hoc score category collapsing as a potential countermeasure against suboptimal rating scale functioning. Findings suggested that different score categories represented the same ability level and were therefore interchangeable. Collapsing these score categories yielded shorter but more functional scales without compromising the psychometric qualities of the original scales. These findings suggest that researchers need to empirically refine their scale lengths rather than uncritically following their conventional measurement practices.