A Comprehensive Framework for Multi-Modal Depression Detection: Integrating Adaptive Fusion, Fairness Regularization, and Explainable AI

Depression poses a major worldwide health burden, necessitating advanced diagnostic tools based on reliable and authentic assessment. While multimodal AI systems offer promising results by integrating diverse data sources, existing approaches often rely on static fusion strategies, exhibit inherent biases, lack interpretability, and show limited generalizability. This paper presents a novel multimodal deep learning framework designed to address these critical limitations. The proposed architecture introduces an adaptive, context-aware, and explainable fusion mechanism, combining a Dynamic Gating Network (DGN) that dynamically adjusts modality contributions with a Multi-Head Attention Network (MHAN) to capture deep inter-modal interactions. In addition, a fairness regularization strategy is incorporated to mitigate algorithmic bias, alongside an Explainable AI (XAI) module to provide transparent and clinically meaningful insights. Quantitative evaluations across the DAIC-WOZ, StudentSADD, and Moodable datasets demonstrate strong and consistent performance, achieving an F1-score of 91.4% with 93.0% accuracy on DAIC-WOZ, 82.0% F1 with 83.7% accuracy on StudentSADD, and 80.3% F1 along with 82.5% accuracy on Moodable. Furthermore, the proposed approach reduces fairness disparities and improves generalizability compared to conventional multimodal baselines. Model explanations were also qualitatively evaluated on all three datasets by three mental-health experts using a 5-point Likert scale in terms of clarity, correctness, and clinical plausibility. Overall, this work represents a significant step toward trustworthy, equitable, and clinically applicable AI systems for robust multimodal depression detection, fostering greater confidence and adoption in mental healthcare.