Machine Learning and Prediction in Psychological Assessment

Abstract. Modern prediction methods from machine learning (ML) and artificial intelligence (AI) are becoming increasingly popular, also in the field of psychological assessment. These methods provide unprecedented flexibility for modeling large numbers of predictor variables and non-linear associations between predictors and responses. In this paper, we aim to look at what these methods may contribute to the assessment of criterion validity and their possible drawbacks. We apply a range of modern statistical prediction methods to a dataset for predicting the university major completed, based on the subscales and items of a scale for vocational preferences. The results indicate that logistic regression combined with regularization performs strikingly well already in terms of predictive accuracy. More sophisticated techniques for incorporating non-linearities can further contribute to predictive accuracy and validity, but often marginally.