Old but Gold or New and Shiny? Comparing Tree Ensembles for Ordinal Prediction with a Classic Parametric Approach

There is a long tradition of modeling ordinal response data with parametric models such as the proportional odds model. With the advent of machine learning (ML), however, the classical stream of parametric models has been increasingly challenged by a more recent stream of tree ensemble (TE) methods extending popular ML algorithms such as random forest to ordinal response data. Despite selective efforts, the current literature lacks an encompassing comparison between the two methodological streams. In this work, we fill this gap by investigating under which circumstances a proportional odds model is competitive with TE methods regarding its predictive performance, and when TE should be preferred. Additionally, we study whether the optimization of the numeric scores assigned to ordinal response categories, as in Ordinal Forest (OF; Hornung, 2019), is worth the associated computational burden. To this end, we further contribute to the literature by proposing the Ordinal Score Optimization Algorithm (OSOA). Similar, to OF, OSOA optimizes the numeric scores assigned to the ordinal response categories, but aims to enhance the optimization procedure used in OF by employing a non-linear optimization algorithm. Our comparison results show that while TE approaches outperformed the proportional odds model in the presence of strong non-linear effects, the latter was competitive for small sample sizes even under medium non-linear effects. Regarding the TE methods, only subtle differences emerged between the individual methods, showing that the benefit of score optimization was situational. We analyze potential reasons for the mixed benefits of score optimization to motivate further methodological research. Based on our results, we derive practical recommendations for researchers and practitioners.