A Novel Method for the In-Depth Multimodal Analysis of Student Learning Trajectories in Intelligent Tutoring Systems

Temporal analyses are critical to understanding learning processes, yet understudied in education research. Data from different sources are often collected at different grain sizes, which are difficult to integrate. Making sense of data at many levels of analysis, including the most detailed levels, is highly time-consuming. In this paper, we describe a generalizable approach for more efficient yet rich sensemaking of temporal data during student use of intelligent tutoring systems. This multi-step approach involves using coarse-grain temporality — learning trajectories across knowledge components — to identify and further explore “focal” moments worthy of more fine-grain, context-rich analysis. We discuss the application of this approach to data collected from a classroom study in which students engaged in a Chemistry Virtual Lab tutoring system. We show that the application of this multi-step approach efficiently led to interpretable and actionable insights while making use of the richness of the available data. This method is generalizable to many types of datasets and can help handle large volumes of rich data at multiple levels of granularity. We argue that it can be a valuable approach to tackling some of the most prohibitive methodological challenges involved in temporal learning analytics