Lightweight Dual-View Feature Fusion for Hand-Object Interaction Recognition

Wrist-worn hand-object interaction (HOI) recog- nition is a critical capability for wearable rehabil- itation systems, assistive technologies,augmented reality and human-computer interaction applica- tions. Compared with fixed external cameras, wrist-worn wearable devices provide user-centered observations that are more suitable for contin- uous real-world interaction monitoring. How- ever, many existing wrist-worn HOI recognition systems still face important challenges, includ- ing incomplete interaction representation caused by single-view observations, viewpoint ambiguity, self-occlusion and the high computational com- plexity of recent deep learning approaches. To address these limitations, this paper proposes a lightweight dual-view framework for hand-object interaction recognition using synchronized palm- view and back-view RGB images acquired from a wrist-worn dual-camera device. The proposed framework employs a shared MobileNetV2 back- bone combined with multi-level feature extraction to jointly capture fine-grained spatial details and high-level semantic representations. To effectively integrate complementary information from differ- ent network depths, a view-specific adaptive fusion mechanism is introduced to dynamically balance intermediate and deep feature representations for each visual stream. The fused dual-view represen- tation is subsequently used for interaction classifi- cation. Experimental evaluation under the Leave- One-Participant-Out (LOPO) cross-subject pro- tocol demonstrates that the proposed framework achieves a mean accuracy of 82.36% and a mean F1-score of 81.65% while maintaining low compu- tational complexity suitable for real-time wearable applications. Ablation studies further confirm the effectiveness of the proposed multi-level feature extraction and adaptive fusion strategy. The pro- posed approach provides an effective balance be- tween recognition performance and computational efficiency for lightweight wearable HOI recognition systems.