Implementing the self-regulated learning structured interview protocol with generative AI: a novel approach for evaluating students’ SRL skills

In recent years, self-report questionnaires and unobtrusive measures of self-regulated learning (SRL) based on digital trace data have been widely used to evaluate students’ SRL skills. However, researchers acknowledged significant limitations in these methods: while students often refuse to fill item-intensive questionnaires, digital trace data often lack the granularity and contextualization required to capture and explain the complexity of learning processes. Interviews (e.g. the Self-Regulated Learning Structured Interview protocol (SRLSI)), another type of instruments, while demonstrated validity and reliability has not been used widely due to its labor-intensive and time-consuming nature on the side of instructors and researchers. To address these challenges, the current research leverages advances in generative artificial intelligence (GenAI) to develop and implement a conversational agent capable of conducting the SRLSI. Grounded in learning sciences, this study evaluates the effectiveness and efficiency of this tool in measuring SRL processes. The findings highlight three key outcomes: first, the tool effectively conducts and concludes SRL interviews; second, it reliably evaluates and categorizes students’ learning strategies; and third, students reported high satisfaction with the quality of the interview discussion, accuracy of the tool’s evaluations, response times, and in part feedback provided. Future research in SRL could combine such tool together with other measures (self-report questionnaires and digital trace data analysis), to provide a more comprehensive understanding of students’ SRL processes. The study concludes with practical recommendations for advancing SRL assessment and emphasizes the potential of multimodal approaches for more accurate and holistic measurement of self-regulated learning.