Answers are grounded in this paper and cite their sources.
How can the health of an English classroom ecosystem be assessed from routine recordings in a way that is accurate, traceable to observable behaviour and inexpensive enough to run inside a school? The question is harder than it looks, because what is taught, how it is delivered and how learners respond are dynamically coupled: the difficulty of the material shapes delivery, and delivery in turn shapes learner behaviour. Single-modality models and shallow feature concatenation capture only fragments of that interplay. To answer the question, we build a multimodal Transformer that encodes textual, acoustic and visual streams through modality-specific front ends, learns their mutual dependence through bidirectional cross-modal attention, and merges them through a learned gate to predict a five-dimensional Ecosystem Health Index whose scoring rubric, rater protocol and entropy-based weighting are fully specified. A gradient-weighted attribution scheme turns attention into modality-decomposed, temporally localised explanations, which show textual features dominating healthy sessions and video-based disengagement cues taking over in degraded ones. A joint distillation and structured pruning pipeline compresses the network from 23.8 M to 2.9 M parameters, an 8.2-fold reduction that retains 88.2% of the teacher model’s RMSE-based performance. On 6,142 sessions recorded in 20 institutions the full model reaches an RMSE of 0.0431 ± 0.0009 and a coefficient of determination of 0.923 ± 0.004 across five random seeds, ahead of unimodal, early-fusion and prior cross-modal baselines under paired significance testing. Error rises by 13.5% to 34.1% once instructors, institutions, provinces, course types or educational levels are held out, and we treat that gap as the honest boundary of the present evidence rather than as noise. Device-level benchmarks on Jetson Nano and Raspberry Pi hardware, together with an explicit responsible-use protocol that rules out punitive teacher evaluation, complete the account.
No takes yet. Share an insight, caveat, or question.
Lv et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: