Decoupled Heterogeneous Multi-Teacher Mutual Distillation for Wearable Human Activity Recognition

Wearable human activity recognition (HAR) requires compact models that can capture temporal dependencies under constrained computation, memory, and energy budgets. Knowledge distillation (KD) offers a practical route to this goal by transferring informative predictions and representations from high-capacity teachers to lightweight students. Existing KD methods for wearable HAR, however, remain limited when structurally different teachers are involved. They commonly rely on homogeneous architectures, use mainly teacher-to-student transfer, and provide limited support for heterogeneous multi-teacher learning. This work proposes a decoupled heterogeneous multi-teacher mutual distillation (DHMMD) framework for wearable HAR. DHMMD performs offline reciprocal optimization between multiple heterogeneous teachers and a compact student, while only the distilled student is retained for inference. The framework combines two complementary paths. Bidirectional decoupled semantic distillation transfers target-class and non-target-class probability information through both aggregated and teacher-specific guidance. Bidirectional contrastive feature distillation aligns intermediate representations across heterogeneous architectures using ensemble-level and pairwise feature supervision. Experiments on HAPT, WISDM, and UCI_HAR show that DHMMD outperforms 21 state-of-the-art KD baselines in terms of F 1 score. Under the MLP-student setting, DHMMD improves the F 1 score by 9.30 percentage points on WISDM. On-device latency measurements on Raspberry Pi 4 and PYNQ-Z2, together with board-power-based energy estimation and memory accounting, further show that the distilled students reduce inference cost relative to the heterogeneous teachers. These results indicate that DHMMD is a practical offline training framework for compact wearable HAR models on mobile and application-processor-class edge platforms.