Facial expression recognition for investigating attention and affective states in synchronous online higher education

Abstract This study aims to identify students’ emotion and attention levels in live online classes using image processing technologies within the field of affective computing and to examine the factors influencing these emotional states. To this end, the Emotion and Attention Tracking System (EATS) platform was developed. The study also investigates the predictive capacity of the data obtained through EATS. An explanatory sequential mixed-methods design was employed. The sample consisted of 40 undergraduate and graduate students. Data were collected through EATS records, questionnaires, observation forms, and semi-structured interviews. Descriptive analysis, Bland–Altman analysis, and content analysis were used to analyze the data. The findings indicate that EATS reliably and accurately detects basic emotions such as surprise, sadness, anger, fear, disgust, happiness, and neutrality, as well as students’ attention levels. However, although disgust is considered a basic emotion, it was measured with lower reliability and showed higher error rates. The factors influencing students’ emotions during live online sessions were mainly related to the instructor, including behavior, tone of voice, communication skills, and technical competence. In addition, peer interaction, learning environment, and technical issues also affected students’ emotional states. Overall, EATS demonstrates a high level of accuracy in recognizing students’ emotions and attention. Affective computing-based platforms such as EATS can serve as effective tools for monitoring learners’ emotional and attentional states in online learning environments. Providing real-time feedback based on these emotional indicators has the potential to enhance instructional effectiveness and increase student engagement.