Measuring Self-Regulated Learning Skills Through Social Interactions in a Teachable Agent Environment

We have developed a learning environment where students teach a computer agent using visual representations, and can monitor the agent's learning progress by asking her questions and having her take quizzes. The system provides self-regulated learning and metacognitive support via dialog-embedded prompts from Betty, the teachable agent, and Mr. Davis, the mentor agent. Our primary goals have been to support learning of complex science topics in middle school classrooms and facilitate development of metacognitive skills to support future learning. In this paper, we discuss methods that we have employed for detecting and characterizing students' behavior patterns from their activity sequences on the system. In particular, we discuss a method for learning hidden Markov models (HMM) from the activity logs. We demonstrate that the HMM structure corresponds to students' aggregated behavior patterns in the learning environment. Overall, the HMM technique allows us to go beyond simple frequency and sequence analyses, such as individual activity and pre-defined pattern counts, instead using exploratory methods to examine how these activities cohere in larger patterns over time. The paper outlines a study conducted in a 5th grade science classroom, presents the models derived from the students' activity sequences, interprets the model structure as aggregate patterns of their learning behaviors, and links these patterns to students' use of self-regulated learning strategies. The results illustrate that those who teach an agent demonstrate better learning performance and better use of metacognitive monitoring behaviors than students who only learn for themselves. We also observed more advanced and focused monitoring behaviors in the students who received metacognitive strategy feedback from the mentor agent while they taught the teachable agent.