The Realiability of Mathematics Portfolio Scores: Lessons From the Vermont Experience

Portfolios are the centerpiece of Vermont's statewide assessment program in mathematics. During the first 2 years of this program, there was only moderate agreement among readers regarding a portfolio's quality. This quality often varied greatly across the five to seven pieces in a student's portfolio. Consequently, scores were not reliable enough to permit reporting student-level results. Increasing the number of readers per piece or the number of pieces per portfolio are not operationally feasible solutions to the score reliability problem.