Using Visualizations of Students' Coding Processes to Detect Patterns Related to Computational Thinking

Abstract Computational thinking (CT) has emerged as a key topic of interest in K-12 education. Children that are exposed at an early age to STEM curriculum, such as computer programming and computational thinking, demonstrate fewer obstacles entering technical fields (Madill et al., 2007). Increased knowledge of programming and computation in early childhood is also associated with better problem solving, decision-making, basic number sense, language skills, and visual memory (Flannery et al., 2013). As a digital competence, coding is explicitly regarded as a key 21st Century Skill, as the "literacy of today," such that its acquisition is regarded as essential to sustain economic development and competitiveness (Bocconi et al., 2016). Therefore, the reliable evaluation of students' coding process data, in context of problem solving tasks that require CT, is of great importance. Prior research has analyzed overall action sequences or code snapshots, but has not interpreted student actions in context of a situation during the problem solving process -- i.e. while creating the solution. A more fine-grained analysis of coding process data is needed, where relevant actions are interpreted as a part of the student's problem solving process. We introduce a novel visualization approach for the analysis of coding process data. This approach has the following benefits: (a) It does not require the definition of process states; (b) It does not accumulate data (either across students or over time) and thus preserves the raw information aspect of the data; (c) It is goal-oriented, by being based on well-defined and measurable performance objectives; (d) It facilitates the definition of specific performance similarity measures for each performance objective (e.g. distance to optimal path or similarity to optimal event sequence), and thus facilitates scoring; (e) It is independent of sequence data length and thus enables time series analysis (e.g. frequency, pauses, etc.) (f) It can visualize each student's performance for each measure as a function of time; and (g) It can be used to inform the feature extraction process by facilitating pattern identification. We present our visualizations of student process data, collected using codeSpark Academy, which introduces children to programming and computational concepts (sequencing, parameters, loops, events, and conditionals) and combines carefully scaffolded puzzles aligned with the curriculum. Our findings clearly show groups of patterns that represent different strategies related to the computational thinking constructs abstraction, decomposition, generalization, modeling, algorithmic thinking, and evaluation.