Student engagement (SE) is central to learning success in virtual environments, and artificial intelligence (AI) models increasingly rely on annotated datasets to estimate it automatically. However, existing datasets vary substantially in how engagement is defined and operationalized. This narrative review examined 31 SE datasets collected in computer-based virtual learning settings, analyzing them across seven dimensions of engagement annotation: sources, modality, timing, temporal resolution, level of abstraction, combination, and quantification. Substantial heterogeneity was observed, with only 3 of the 31 datasets explicitly grounding their annotations in psychometrically validated or theoretically established engagement constructs. These inconsistencies have important implications for AI model development. For example, mismatches between the modality used for annotation and that used for model training may reduce robustness, and divergent quantification schemes (binary, ordinal, or continuous) restrict cross-dataset comparability and transferability. By explicitly linking dataset design decisions to risks affecting model validity, generalization, and interpretability, this review highlights the need for stronger construct grounding, clearer reporting of annotation protocols, and greater standardization to support more reliable and transferable engagement-aware AI systems.
Khan et al. (Wed,) studied this question.