Multimodal Deep Fusion of EEG–FNIRS and Behavioral Signals for Predicting Depression Severity Via Graph Neural Networks

One of the leading causes of impairment in the world is major depressive disorder (MDD), and a correct diagnosis and treatment plan depend on an accurate assessment of the disorder’s severity. Modern methods frequently use unimodal signals, which fall short in capturing the complex neurobehavioral foundations of depression. In order to overcome this limitation, we offer a multimodal deep fusion framework using graph neural networks (GNNs) that combines behavioral inputs, electroencephalography (EEG), and functional near-infrared spectroscopy (fNIRS) to predict the severity of depression. To illustrate physiologically significant interactions between and across modalities, we specifically create correlation-weighted graphs. This enables the network to learn multimodal embeddings that are both compact and discriminative. The framework is evaluated on a dataset comprising 100 participants (50 diagnosed with MDD and 50 healthy controls), integrating multimodal data from 32-channel EEG, 16-optode fNIRS, and behavioral assessments derived from cognitive tasks, thereby ensuring diversity in neurophysiological and behavioral representations. The suggested framework routinely outperforms unimodal models and traditional machine learning benchmarks in both regression and classification tasks, according to extensive experiments using a dataset of 100 individuals. Furthermore, the clinical relevance of this technique is highlighted by strong correlations between verified clinical metrics and expected scores. The results confirm the effectiveness of multimodal GNN-based fusion as a reliable and objective tool for improving computational psychiatry and strengthening the assessment of depression severity.