Multi-modal voice feature fusion for the diagnosis of Parkinson’s disease and atypical Parkinsonian syndromes using custom deep learning

The proposed study addresses the gap between traditional clinical diagnosis of Parkinsonian disorders and the emerging potential of deep learning based voice analysis as a non-invasive diagnostic tool. Considering that Parkinson’s Disease (PD) significantly affects speech production, this work comprehensively investigates speech as a reliable biomarker using a multi-representation learning framework. Two benchmark datasets are utilized in this study: a binary dataset (HC vs. PD) and a multiclass dataset (HC, PD, MSA, PSP). Speech signals are preprocessed and transformed into complementary time–frequency representations, including Mel spectrograms, spectral contrast, chroma features, and enhanced STFT variants. A multi-branch custom CNN is proposed to fuse these features, achieving strong performance with ROC–AUC scores of 0.955 ± 0.018 (HC) and 0.963 ± 0.013 (PD), and overall accuracy exceeding 94% in binary classification, and 0.910 ± 0.016 AUC with 91.0 ± 1.7% accuracy in multiclass classification. Ablation analysis exhibits consistent performance gains with feature fusion, confirming the complementary nature of acoustic representations, while paired t -tests establish statistical significance of improvements ( p < 0.05 , p < 0.01 ). Additionally, Grad-CAM-based explainability reveals that the model focuses on clinically relevant spectral regions associated with vocal impairments. These findings validate that multi-representation deep learning provides a robust, interpretable, and scalable framework for early and non-invasive detection of Parkinsonian disorders.