Behavioural telemetry is a weak substitute for in-course assessment: predicting engineering students’ examination performance from digital laboratory process data

Learning analytics promises early identification of struggling students, but evidence comes mainly from learning-management-system counts rather than authentic laboratory telemetry. We tested the predictive value of fine-grained process traces in a first-year digital electronics laboratory. The open Educational Process Mining dataset contains 115 engineering undergraduates observed across six simulator sessions (230,318 records) and 93 students with examination outcomes. Under a pre-specified, leakage-aware protocol—nested repeated cross-validation, pipeline-internal preprocessing, a 5 × 3 feature-group-by-model ablation matrix, and bootstrap confidence intervals—we predicted below-median final-examination performance. The pre-specified behavioural logistic model, using eleven features spanning volume, engagement, and interaction intensity, achieved an AUC of 0.617 (95% CI 0.491–0.737), not reliably above chance; the best non-pre-registered behavioural model reached 0.648. Four in-course-grade features achieved an AUC of 0.764 (0.644–0.854), the best matrix cell. Behaviour underperformed grades ( Δ AUC − 0.147 , 95% CI − 0.263 to − 0.028 ), and adding telemetry to grades produced no detectable improvement ( Δ AUC − 0.040 , − 0.083 to 0.006 ). With one session, behavioural prediction was below chance and plateaued near 0.62 from session four, while the single grade available at session two already outperformed six sessions of telemetry; its signal concentrated in exercise coverage and workflow and partly reflected a small lower-participation group that often missed the examination. Full telemetry predicted non-participation well (AUC 0.872), but exercise coverage alone matched it (0.879). Continuous-score regression also favoured grades as standalone predictors (best MAE 21.29 vs. 22.38), although nonlinear unions modestly improved error. In this setting, telemetry is primarily a coarse disengagement flag; the course’s assessment loop remains the more trustworthy signal for participating students.