Effectiveness of an AI-Based Adaptive Assessment System for Conceptual Mastery in Grade 8 Mathematics: Protocol for a Cluster-Randomised Trial with Simulation-Based Design Checks

Adaptive assessment systems promise to diagnose each learner’s gaps and respond with targeted tasks and feedback, yet trials rarely show that they improve independently measured conceptual understanding beyond what equal time on conventional digital practice achieves. This protocol describes a two-arm cluster-randomised trial in schools that follow the Cambridge Lower Secondary Mathematics framework. Grade 8 mathematics teachers will be randomised within schools to AIMAS-8, a system that combines a concept-level dynamic Bayesian learner model, prerequisite-aware item selection and expert-authored misconception feedback, or to the same items delivered in a fixed order with worked solutions. Both arms receive twelve 30 min sessions on linear relationships. The primary outcome is an independently developed conceptual mastery test at week 7; secondary outcomes are transfer, retention at week 11 and moderation by baseline achievement. Design checks are reported: 46 recruited teacher clusters (approximately 41 expected to remain after cluster attrition) and about 1500 expected analysed students provide 80% power for a 0.25 SD difference under the stated planning assumptions; preliminary simulation checks retained a nominal Type I error of 0.050 and recovered known effects without appreciable bias; and the learner model did not classify mastery more accurately than a fixed sequence, which led to a coverage safeguard. The planning sequence is approximately 2300 students invited, 85% consent, 15% student-level attrition and 10% teacher-cluster attrition. No participant data have been collected.