EngageSense: A Hybrid Approach for Real Time Engagement Detection for Virtual Classrooms

Advancements in digital education have revolutionized traditional learning environments, driving the widespread adoption of virtual and hybrid classrooms. Engagement, a vital factor for effective learning, necessitates continuous monitoring and assessment to optimize outcomes. This study introduces EngageSense, a hybrid real-time engagement detection system leveraging facial biometrics, computer vision, and deep learning. First, a new dataset is created via user eye images taken from webcam of laptop. Then, Dlib's HOG + Linear SVM for face detection, a CNN model trained on 4,453 eye images dataset(classified into left, right, and center gaze directions), and OpenPose MobileNetV1 for body pose estimation are used. By fusing gaze direction (99.50% accuracy) and pose features, EngageSense classifies engagement into three levels: fully engaged, partially engaged, and not engaged with an accuracy of 90%. By providing actionable real-time insights, EngageSense empowers educators to foster meaningful interactions and enhance learning experiences in virtual environments.