Spectral Multi-Representation Fusion for Audio Deepfake Detection

Audio deepfake detection systems often achieve excellent internal validation performance but fail to generalize under real-world inference conditions involving synthetic speech generated with previously unseen AI tools. To address this limitation, this work proposes the Spectral Multi-Representation Fusion (SMRF) framework, which integrates multiple spectral representations and decision-level fusion strategies to improve robustness under cross-domain conditions. Additionally, a Stability-Aware Multi-Metric Selection (SAMMS) strategy is introduced to select architectures by jointly considering predictive performance and cross-representation stability. The proposed framework was evaluated using four spectral representations (log-magnitude spectrogram (LOG), Mel spectrogram (MEL), Discrete Wavelet Transform (DWT), and Constant-Q Transform (CQT)) combined with multiple convolutional architectures and complementary voting strategies. The experiments revealed that isolated models exhibiting validation metrics above 95% may still produce very poor synthetic-audio detection rates during external inference (even lower than 10%). In contrast, fusion-based strategies substantially improved robustness by exploiting complementary synthetic evidence across spectral domains. The results also demonstrated that both the voting strategy and the SAMMS stability parameter λ strongly affect the final behavior of the system. In particular, hybrid fusion using One-Hard Voting with two architectures selected using λ≥0.25 achieved the best balance between synthetic-audio detection and real-audio preservation, outperforming individual models under cross-domain inference conditions, with detection rates close to 75% for both synthetic and real audio. These findings suggest that stability-aware fusion strategies constitute a promising direction for improving robustness in realistic audio deepfake detection scenarios.