An IRT Approach to Constructing and Scoring Pairwise Preference Items Involving Stimuli on Different Dimensions: The Multi-Unidimensional Pairwise-Preference Model

This article proposes an item response theory (IRT) approach to constructing and scoring multidimensional pairwise preference items. Individual statements are administered and calibrated using a unidimensional single-stimulus model. Tests are created by combining multidimensional items with a small number of unidimensional pairings needed to identify the latent metric. Trait scores are then obtained using a multidimensional Bayes modal estimation procedure based on a mathematical model called MUPP, which is illustrated and tested here using Monte Carlo simulations. Simulation results show that the MUPP approach to test construction and scoring provides accurate parameter recovery in both one- and two-dimensional simulations, even with relatively few (say, 15%) unidimensional pairings. The implications of these results for constructing and scoring fake-resistant personality items are discussed.