Exploring subgroup heterogeneity in online and in-person learning outcomes using causal forest

This study examines the heterogeneous effects of online versus face-to-face (F2F) learning on student outcomes, using machine learning to uncover subgroup differences rather than focusing on average effects. Analyzing data from 2210 students, we apply a causal forest method to estimate individualized and subgroup-level effects, considering factors like prior academic performance, specialization, and instructor differences. Nine out of 35 subgroups, defined based on baseline student characteristics (e.g., academic performance, major, and instructor), exhibit significantly heterogeneous effects, with four remaining distinct after multiple testing corrections. For instance, students with lower secondary school scores benefit from online learning in terms of exam scores, while those with higher scores and certain specializations, such as Computer Engineering, experience negative effects. These findings underscore that evaluations of teaching modalities must account for students’ heterogeneous characteristics rather than assuming uniform impacts. By identifying which students benefit most from each mode, this research offers realistic insights for optimizing learning outcomes and addressing inequities in diverse educational contexts. These insights challenge one-size-fits-all approaches to digital education and highlight the need for modality selection frameworks that consider the potential heterogeneity. Mainly, our results demonstrate that discipline-specific effects are non-trivial, with quantitative outcomes revealing significant performance declines in applied fields (e.g., Computer Engineering) under online instruction—suggesting that hands-on disciplines may benefit from face-to-face components to preserve learning quality.