Differentiable Models for Music Signal Reconstruction, Adaptation, and Synthesis

Nuanced variations in pitch, loudness, and timbre are key components of musical expression and the perceived naturalness of musical sounds. Recent advances in deep learning have enabled powerful data-driven methods for manipulating these attributes in existing recordings and for synthesizing novel sounds. However, many deep learning approaches remain limited in terms of controllability of the output, their reliance on high-quality training data, and computational cost. In this thesis, we explore differentiable models as one way of addressing these limitations. By explicitly integrating physical, perceptual, or signal-based structure into deep learning pipelines or loss functions, these models can provide gradients with respect to musically interpretable parameters. At the same time, they allow for incorporating inductive biases grounded in audio signal processing, Music Information Retrieval, or psychoacoustics, and support computationally efficient implementations. The contributions of this thesis are organized around three high-level tasks: reconstruction, adaptation, and synthesis of monophonic music signals. First, reconstruction addresses the problem of recovering corrupted or missing tone attributes. In this context, we introduce a differentiable model inspired by the vocal tract that enables the reconstruction of natural timbre characteristics from low-quality singing voice signals recorded with larynx microphones. The vocal tract model supports learning from limited training data while reliably preserving the original pitch. Second, adaptation focuses on modifying individual tone attributes without affecting others. Within representative case studies, we propose differentiable cost functions for intonation and timbre adaptation that integrate perceptual and musical models. The formulation enables gradient-based optimization that accounts for interactions between voices in multi-track recordings. Finally, synthesis refers to the task of creating an audio signal from a disentangled parametric representation of pitch, loudness, and timbre. Using wind instrument resynthesis as an application scenario, we introduce a physically inspired instrument model based on differentiable pulsetable synthesis that allows for controllable and computationally efficient signal generation. Across these tasks, we analyze strengths and limitations of differentiable models, with particular emphasis on the interplay between model structure and loss function. Complementing the application-specific contributions, we examine how loss function behavior with respect to individual tone attributes affects the convergence of deep learning systems with and without model-based components. In particular, we investigate the widely used multi-scale spectral loss and show that a favorable configuration leads to a smoother loss function with respect to pitch-related parameters. Together, these insights inform the practical design of model-based deep learning systems and contribute to more controllable, data-efficient, and computationally lightweight methods for music processing and related domains.