Making Sense of Common Test Items That Do Not Get Easier Over Time: Implications for Vertical Scale Designs

This study focuses on an instance in which the mean grade-to-grade scale scores on a vertical scale showed evidence of common test items that do not get easier from one grade to the next. The issue was examined as part of a 2-day workshop in which participants were asked to predict the growth on all linking items used in the construction of vertical scales in math and reading. It was found that these predictions were only weakly correlated to the empirical changes overall. The participants seldom found legitimate rationales for excluding reversal items. Results indicate that a reversal will almost never be sufficient as a rationale for the exclusion of a linking item from the calibration of a vertical scale. Greater attention needs to be given to definition of the construct over which growth is to be interpreted and how this influences the selection of linking items.