Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset

Generating musical audio directly with neural networks is notoriously because it requires coherently modeling structure at many different. Fortunately, most music is also highly structured and can be as discrete note events played on musical instruments. Herein, we that by using notes as an intermediate representation, we can train a of models capable of transcribing, composing, and synthesizing audio with coherent musical structure on timescales spanning six orders of (~0. 1 ms to ~100 s), a process we call Wave2Midi2Wave. This large in the state of the art is enabled by our release of the new MAESTRO (MIDI and Audio Edited for Synchronous TRacks and Organization) dataset, of over 172 hours of virtuosic piano performances captured with fine (~3 ms) between note labels and audio waveforms. The networks and the together present a promising approach toward creating new expressive interpretable neural models of music.