Synthesizing Results from the Trial State Assessment

Using data collected under the Trial State Assessment (TSA) of the National Assessment of Educational Progress (NAEP), this article describes and illustrates a two-stage statistical model for investigating state-to-state variation in mathematics achievement. At the first stage, within each state, a two-level hierarchical linear model is estimated via maximum likelihood. At the second stage, results are combined across states using Bayesian estimation implemented via the Gibbs sampler. The results reveal considerable state-to-state heterogeneity in mathematics proficiency, but most heterogeneity is explainable on the basis of covariates defined on students, teachers, and schools. The findings suggest that interest in state comparisons might productively focus on state differences in policy-relevant correlates of proficiency rather than on state differences in mean proficiency. The analytical approach can be applied in other cases where data are dense at the lower level of a hierarchy but thin at the higher level.