Trimodal Fusion for Depression Screening Without Data Leakage: Evaluation of Symmetric Bidirectional Cross-Attention on the DAIC-WOZ Dataset (Preprint)

BACKGROUND Although the early and accurate identification of major depressive disorder—a leading cause of global disability—is a paramount clinical priority, screening still relies on subjective questionnaires vulnerable to under-reporting, creating an urgent need for objective, passively collected multimodal biomarkers. OBJECTIVE To expose data leakage in prior research for correcting flawed benchmarks, develop a stable tri-modal fusion model with symmetric bidirectional cross-attention on the Distress Analysis Interview Corpus - Wizard of Oz (DAIC-WOZ) dataset, assess the performance of ChatGPT o1 pro, a state-of-the-art model at the time of research, in transcript classification and conduct ablations on modality combinations, pooling strategies, and utterance overlaps to offer foundational insights for the research community. METHODS This study used the DAIC-WOZ dataset (n=189; 56 depressed, 133 non-depressed) of audio-visual-text interviews. Data was segmented into five-utterance windows with 0-4 utterance overlaps. We extracted audio features with Wav2Vec 2.0 and visual features (gaze, pose, etc.) with a bidirectional long short-term memory. ChatGPT o1 pro provided binary depression labels from transcripts. A tri-modal fusion model's performance was evaluated using accuracy, precision, recall, and F1-score at segment and participant levels. We also performed ablations on modalities, pooling, and overlaps, and replicated prior flawed splits to check for data leakage. RESULTS Replicating the flawed methodology from a study that garnered considerable attention produced inflated accuracy (96.6%) from the audio modality alone, whereas enforcing a participant-disjoint split caused performance to collapse to 63.2% accuracy, confirming data leakage. Our architecture successfully stabilizes a symmetric cross-attention block—a powerful fusion method discarded for its inherent instability—by strategically integrating it with attentive pooling, modality dropout, and a gating unit. This approach achieved a participant-level accuracy of 86.8% and a macro-averaged F1-score of 0.84. While text was the strongest single modality (81.6% accuracy), adding speech and vision with modality dropout raised accuracy by 5.2 percentage points. Attentive pooling lifted the macro F1-score by 23.2 percentage points over mean pooling, and a one-utterance overlap was found to maximize performance, as larger overlaps degraded it. CONCLUSIONS This work exposes critical flaws in widely cited research stemming from participant-level data leakage, which inflated reported accuracies to 96% while failing to generalize, risking misdiagnosis in premature clinical use and misleading community benchmarks. We demonstrate that through deliberate architectural choices, a symmetric bidirectional cross-attention mechanism—previously abandoned for its instability—can be successfully applied, reinstating it as a viable strategy for modest clinical datasets. Furthermore, our ablation studies correct the prevailing view that more utterance overlap is advantageous. By providing the first comprehensive analysis of multiple factors—including modality combinations, pooling strategies, and utterance-window overlap—at both the participant and utterance levels, our work offers foundational insights for developing objective, passively collected biomarkers to augment depression screening.