Translating musical content into coherent natural language descriptions is a difficult cross-modal generation problem, mainly because acoustic representations entangle multiple musical attributes and a wide semantic gulf separates audio from text. We present a framework that brings together acoustic feature decoupling and large-scale pre-trained language models for music captioning. At its core, a variational autoencoder-based module factorises audio representations into three semantically distinct subspaces-content, style, and emotion-guided by mutual information minimisation together with orthogonality constraints. A multi-granularity alignment mechanism then bridges the decoupled acoustic features and linguistic representations through both global and local contrastive learning. The pre-trained language model decoder, adapted via prefix tuning, retains its accumulated linguistic knowledge while accepting multimodal conditioning. Across three benchmarks-MusicCaps, Song Describer, and LP-MusicCaps-the framework delivers competitive performance relative to recent baselines, with relative gains of 18.9% in BLEU-4 and 4.4% in BERTScore over CLAP-Cap on MusicCaps. Ablation studies confirm that each component contributes meaningfully to generation quality, while human evaluation validates superior fluency, relevance, informativeness, and diversity of generated descriptions.
Jiaying Yu (Tue,) studied this question.