Enhancing literacy and numeracy through teacher dashboards and digital personalised learning: an A/B/C test in Kenyan early years education

Although research indicates that Digital Personalised Learning (DPL) can improve aspects of early-years literacy and numeracy in low- and middle-income countries (LMICs), it is necessary to determine whether teacher-facing learning analytics dashboards further impact education outcomes. Specifically, there is a need to evaluate different dashboard designs to determine which design features best support pedagogical decision-making, given that evidence on how such dashboards are used in LMICs remains limited. This study is the first to explore how different dashboards, designed to present teachers with learners’ foundational literacy and numeracy competency levels, can impact device usage, learning outcomes and teachers’ evaluation of learner progress. An A/B/C test in 4487 Kenyan schools compared three randomly assigned school partitions: no dashboard, an algorithm-generated List Dashboard and an algorithm-generated Grouped Dashboard. Impact was evaluated based on learners’ usage of DPL and its impact on literacy and numeracy outcomes. The alignment between algorithm-generated rankings and teachers’ ratings of these rankings was also investigated to determine how practitioners make sense of different data presentations. Findings demonstrate that learners in the Grouped Dashboard partition used the DPL tool more than those in the other two partitions. They also scored statistically significantly higher on two literacy summative test units, but not on numeracy test units. However, despite its potential benefits, the Grouped Dashboard was perceived as being less accurate in determining learner level according to teacher ratings. This research is significant in bridging the gap between teacher-facing dashboard design and its intended impact on teacher sensemaking, learner device usage and learning outcomes in DPL environments in LMICs. Outcomes rigorously inform future design decisions on teacher-AI collaboration and highlight the potential of A/B/n testing to consider cultural and regional factors in technology-supported instruction.