Fingering optimization method of an intelligent piano sparring system based on multimodal hidden Markov model and reinforcement learning

The automatic generation of piano fingering is the core problem in an intelligent sparring system. The actual playing process is influenced by the note sequence structure, finger physiological constraints, and individual playing habits at the same time. It is often difficult for traditional methods to strike a balance between finger rationality and overall comfort. Focusing on this problem, a Decision-constrained Multimodal Hidden Markov Reinforcement Learning (DMHRL) model is proposed. In this model, the multi-modal note features are used to construct the time sequence representation, and the prior knowledge of fingering is introduced into the Hidden Markov Model (HMM) to form a decision constraint, which is used to screen the locally feasible fingering paths. Combined with the Reinforcement Learning (RL) strategy based on Q-learning, the fingering comfort is optimized on the whole level. The experimental results show that this method is obviously superior to many baseline methods in fingering accuracy, playing comfort, and turning index. The fingering accuracy reaches 88.7%, the playing comfort is 0.81, and the turning index is reduced to 28. Statistical tests based on five independent runs showed that the differences between DMHRL and all baseline models on the aforementioned metrics were statistically significant, with p < 0.05. Ablation experiments demonstrated that multimodal representation, decision constraints, and the RL mechanism played complementary roles. The proposed method showed good applicability to offline piano score fingering optimization, while its response efficiency and dynamic adaptability in real-time interactive piano accompaniment environments require further validation.