An implementation of argument-based validation for assessing college major preferences with a hybrid of Likert-rating and forced-choice formats

Choosing a college major is a significant career decision. The College Major Preference Assessment (CMPA) helps with this process by using three rounds of Likert-scale ratings to eliminate majors that are not preferred, followed by four rounds of forced-choice questions to narrow down an individual’s top three choices from a list of 50 options. This study used argument-based validation to evaluate whether the CMPA’s design effectively serves its purpose. Researchers examined the assessment’s claims, inferences, warrants, assumptions, supporting evidences, and rebuttals. Data collected for Psychology and Education majors were analyzed using latent trait models, revealing psychometric qualities that match the goals of each round of assessment. These findings were also confirmed by a separate, independent group. Additionally, the study demonstrates that argument-based validation can be flexibly applied to assessments with mixed formats.