Using multimodal learning analytics as a formative assessment tool: Exploring collaborative dynamics in mathematics teacher education

Abstract Background Traditionally, understanding students' learning dynamics, collaboration, emotions, and their impact on performance has posed challenges in formative assessment. The complexity of monitoring and assessing these factors have often limited the depth and breadth of insights. Objectives This study aims to explore the potential of multimodal learning analytics as a formative assessment tool in math education. The focus is on discerning how collaborative discourse behaviours and emotional indicators interplay with lesson evaluation performance. Methods Using undergraduate students' multimodal data, which includes collaboration data, facial behaviour data, and emotional data, the study explored the patterns of collaboration and emotion. Through the lens of multimodal learning analytics, we conducted exploratory data analysis to identify meaningful relationships between specific types of collaborative discourse, facial expressions, and performance indicators. Moreover, the study evaluated a machine learning model's potential to predict target learning outcomes by integrating data from multiple channels. Results The analysis revealed key features from both discourse and emotion data as significant predictors. These findings underscore the potential of a multimodal analytical approach in understanding students' learning process and predicting outcomes. Conclusions The study emphasizes the importance and feasibility of a multimodal learning analytic approach in the context of math education. It highlights the academic and practical implications of such an approach, along with its limitations, pointing towards future research directions in this area.