Dance and Martial Arts Fusion Motion Generation Technology Based on Multimodal Fusion and Transformer

This paper presents a multimodal motion generation framework integrating high-frequency textile strain sensing (TSS) arrays with a Spatially Modulated Co-Attention (SMCA) Transformer for real-time dance and martial arts fusion. A contrastive multimodal alignment (CMA) mechanism is employed to achieve robust cross-modal feature integration among strain sensor signals, audio rhythms, and semantic embeddings. The SMCA architecture decouples and fuses spatiotemporal dual-stream features, while differentiable biomechanical constraints—including bone length consistency and zero-moment-point balance—ensure physical plausibility of the generated motion trajectories. Experimental validation demonstrates a motion reconstruction accuracy (MRA) of 0.942–0.965, a bone length consistency loss reduced to 0.15, and a high-efficiency frame processing rate (FPR) of 124.6 fps. By framing the framework as a sensor-array-based signal acquisition and propagation system with physics-constrained output, the study provides an engineering-oriented methodology for real-time generation, spatiotemporal signal alignment, and wave-propagation-inspired trajectory analysis. The approach offers potential applications in immersive human-computer interaction, virtual performance systems, and digital cultural heritage preservation.