Experimental study demonstrates high-accuracy multimodal fingering recognition and tailored training in piano learners, indicating enhanced technique standardization and practice efficiency.
Intelligent recognition of piano fingering requires fine-grained modeling of hand motion, note events, and acoustic characteristics. Existing fingering recognition algorithms often suffer from incomplete feature extraction, weak antiinterference capability, and insufficient integration with personalized practice guidance. This study proposes a multimodal fingering recognition and practice-plan generation framework based on image, MIDI, and audio data. Highdefinition video captures hand motion trajectories, MIDI data provide note position and dynamics, and audio signals supply spectral and timbral cues. HRNet extracts 21 hand key points, Word2Vec-CBOW encodes MIDI sequences, and MFCC-based features represent audio signals. A Dynamic Channel Attention module performs multimodal feature fusion, and an improved Transformer with Position-Aware Encoding captures irregular temporal dependencies in fingering sequences. A personalized practice-generation model further combines recognition outputs, learner profiles, and teaching rules to dynamically adjust practice content, difficulty, and pace. Experiments using the MAESTRO dataset and real performance data show that the method achieves 96.2% Finger Accuracy and 94.5% Direction Accuracy, outperforming HMM, Bi-LSTM, and standard Transformer baselines. After personalized practice intervention, learners’ fingering standardization improves by 28.3% and practice efficiency by 32.1%. The framework supports multimodal signal processing, temporal sequence modeling, and intelligent music-training systems.
No takes yet. Share an insight, caveat, or question.
J. Y. Dong (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: