IICL-DTA : A Contrast Framework for Automatic Depression Detection via Text and Audio

Automatic Depression Detection (ADD) methods, which can assist physicians in the early diagnosis and analysis of depression, have demonstrated promising performance. However, the current ADD methods face two primary challenges: (1) the ADD methods struggle to effectively capture the subtle differences between depressive and non-depressive samples. These minute variations in depressive cues make it difficult for models to distinguish between depressive and non-depressive samples. (2) Due to the privacy and sensitivity associated with depression, collecting data for ADD is challenging, resulting in small dataset sizes. This low-resource issue has become one of the main challenges for this task. In this paper, we propose an intra- and inter-contrastive learning framework (IICL-DTA) for supervised multimodal automatic depression detection. We employ BERT (Bidirectional Encoder Representations from Transformers) and SpeechFormer++ (a hierarchical efficient framework for paralinguistic speech processing) to extract textual and audio features, respectively. We then employ a projection layer to produce positive and negative feature pairs, which are incorporated into the intra- and inter-contrastive learning loss function. Furthermore, we design a multimodal auxiliary loss based on mixup features of text and audio to mitigate the low-resource data issue. This combined intra- and inter-contrastive learning approach encourages similar fused text–audio features to cluster while pushing dissimilar ones apart. We evaluate our method on three datasets: English DAIC-WOZ, E-DAIC, and Chinese CMDC datasets. Our method achieves F1-Scores (D) of 0.90, 0.80, and 0.97 on these datasets, respectively, outperforming previous methods. These results demonstrate the effectiveness and superiority of our approach.