Eliminating False Positives in Adaptive Learning via Multi-Layer Progressive Diagnosis

Single-layer Bayesian Knowledge Tracing (BKT) suffers from an under-examined limitation: in multiple-choice contexts, the guess rate g reaches 0.36—students who have not mastered a knowledge component (KC) still answer correctly 36% of the time. The resulting posterior false positive rate—the proportion of correct answers attributable to guessing rather than mastery—ranges from 18% to 54% depending on group mastery prevalence. In education, false positives carry asymmetric cost: advancing past an unmastered prerequisite creates cascading downstream failure. We propose a three-tier progressive diagnostic architecture (S/R/A: Standard → Reverse → Applied) grounded in Variation Theory, which layers assessments of increasing cognitive demand to filter out shallow mastery. Calibrated on three independent public datasets totaling 13.7M interactions (ASSISTments, EdNet-KT1, KDD Cup 2010), then evaluated via BEAGLE simulation parameterized by each dataset’s empirically derived g and s values, the architecture suppresses false positives by 97% in high-guess scenarios. The theoretical false positive rate under conditional independence is g1g2g3 ≈ 0.65%; no three-layer pass-through was observed in 105 simulation trials. This safety margin is achieved at 35% fewer questions on average and an acceptable increase in false negatives. Calibrated BKT parameters and simulation results are released on Zenodo.