SentHYMNent: An Interpretable and Sentiment-Driven Model for Algorithmic Melody Harmonization

Music composition and analysis is an inherently creative task, involving a combination of heart and mind. However, the vast majority of algorithmic music models completely ignore the "heart" component of music, resulting in output that often lacks the rich emotional direction found in human-composed music. Models that try to incorporate musical sentiment rely on a "valence-arousal" model, which insufficiently characterizes emotion in two dimensions. Furthermore, existing methods typically adopt a black-box, music agnostic approach, treating music-theoretical and sentimental understanding as a by-product that can be inferred given sufficient data. In this study, we introduce two major novel elements: a nuanced mixture-based representation for musical sentiment, including a web tool to gather data, as well as a sentiment- and theory-driven harmonization model, SentHYMNent. SentHYMNent employs a novel Hidden Markov Model based on both key and chord transitions, as well as sentiment mixtures, to provide a probabilistic framework for learning key modulations and chordal progressions from a given melodic line and sentiment. Furthermore, our approach leverages compositional principles, resulting in a simpler model that significantly reduces computational burden and enhances interpretability compared to current state-of-the-art algorithmic harmonization methods. Importantly, as shown in our experiments, these improvements do not come at the expense of harmonization quality. We also provide a web app where users can upload their own melodies for SentHYMNent to harmonize.