Efficient student behavior detection in classroom videos using a keyframe aware multitask transformer

Student Behavior detection in classroom videos has significant benefits for maintaining classroom discipline, monitoring students’ reactions, and evaluating teaching methods. Recent developments in deep learning techniques have enabled robust automated systems across diverse domains. However, despite the widespread adoption of these techniques in industrial application, automated classroom student behavior detection remains an evolving research area in terms of computational efficiency and an effective temporal modeling. Hence, this study attempts to design a transformer-based model with a lightweight, learnable keyframe selection module that learns to globally rank and select the top-K frames per video sequence in an end-to-end manner using downstream classification objectives. For better semantic representations of video clips, the CLIP model was employed, and a customized transformer model with two classification heads was used for behavior prediction and anomaly classification. The model was assessed with spatial- and spatiotemporal-based models on the EduNet dataset, and an ablation experiment illustrated the impact of the keyframe selection module. The proposed model achieves 91% accuracy in behavior detection, demonstrating performance comparable to existing methods while maintaining improved computational efficiency. The proposed system contributes to the education domain by maintaining classroom ecology.