Integrating multimodal feedback to support the training of teaching competences in educational simulations

Educational simulations provide safe environments for the training of hands-on skills. Teacher Moments (TM) is a web platform that delivers classroom simulations for novice teachers. Nevertheless, current simulations lack automated feedback, which can lead to the internalisation of incorrect approaches. In response to this need, we applied speech analysis and the automatic identification of teacher strategies to two datasets from classroom simulations. For speech analysis, we collected a comprehensive dataset of 5563 audio recordings from 361 novice teachers. This analysis revealed that 77.69% of participants showed significant changes in at least one speech metric across the simulation phases. To identify teacher strategies, we assessed six open-weights Large Language Models (LLMs): Gemma 3, GPT-2, Llama 3, Mistral 3, Phi 4, and Qwen 3. For this evaluation, we utilised a dataset of 1,201 unique responses from 494 novice teachers. Qwen 3 achieved the highest performance, with a balanced accuracy of 0.82. Although promising, the performance of LLMs in identifying teacher strategies can be further improved. Finally, we combined both analyses into a flexible architecture that supports the integration of future models to provide multimodal feedback. We incorporated these components into TM to enhance the learning experience in classroom simulations.