Addressing tonal weaknesses in neural audio codecs through synthetic data augmentation and balanced training

Neural audio codecs (NACs) achieve strong perceptual quality at low bitrates but consistently struggle with harmonic-rich, sustained tonal content such as bells, glockenspiel, and piano. Prior work has used synthetic tonal augmentation mainly to improve embedding quality for evaluation; we instead treat it as a first-class training methodology that directly targets codec reconstruction fidelity. This paper presents Project Chimera, a training framework for tonal-robust NACs, instantiated as a 48 kHz codec with 32 codebooks at 30 kbps. The framework introduces three coordinated components: (1) a physically-inspired tonal synthesis generator modelling inharmonic partial structures and resonant decay beyond simple sinusoids; (2) a difficulty-aware curriculum sampling strategy that progressively increases tonal complexity during training; and (3) Exponential Moving Average (EMA) codebook stabilisation with quantisation-capacity analysis that characterises and prevents Residual Vector Quantisation (RVQ) collapse under tonal training. In limited-scale experiments on a 720-hour dataset, Project Chimera achieves 0.51 mel distance and 4.42 ViSQOL at 30 kbps on these objective metrics, outperforming the Descript Audio Codec (DAC) at comparable bitrates. A listening test with $$n=22$$ trained listeners shows subjective gains of approximately $$+0.47$$ MOS overall and $$+0.9$$ MOS on tonal material; the 95% confidence intervals for the overall, tonal, and general-music comparisons separate from DAC, while speech and environmental scores remain comparable. Per-phase checkpoints of the curriculum run show tonal fidelity, codebook entropy, and subjective quality improving monotonically as tonal complexity is staged in. The objective results come from a limited-scale training run; large-scale objective replication remains future work. Code and model weights will be released upon acceptance.