Assessing expertise in introductory physics using categorization task

The ability to categorize problems based upon underlying principles, rather than surface features or contexts, is considered one of several proxy predictors of expertise in problem solving. With inspiration from the classic study by Chi, Feltovich, and Glaser, we assess the distribution of expertise among introductory physics students by asking three introductory physics classes, each with more than a hundred students, to categorize mechanics problems based upon similarity of solution. We compare their categorization with those of physics graduate students and faculty members. To evaluate the effect of problem context on students' ability to categorize, two sets of problems were developed for categorization. Some problems in one set included those available from the prior study by Chi et al. We find a large overlap between calculus-based introductory students and graduate students with regard to their categorizations that were assessed as ``good.'' Our findings, which contrast with those of Chi et al., suggest that there is a wide distribution of expertise in mechanics among introductory and graduate students. Although the categorization task is conceptual, introductory students in the calculus-based course performed better than those in the algebra-based course. Qualitative trends in categorization of problems are similar between the non-Chi problems and problems available from the Chi study used in our study although the Chi problems used are more difficult on average.