Two Birds With One Stone: Knowledge-Embedded Temporal Convolutional Transformer for Depression Detection and Emotion Recognition

Depression is a critical problem in modern society that affects an estimated 350 million people worldwide, causing feelings of sadness and a lack of interest and pleasure. Emotional disorders are gaining interest and are closely entwined with depression, because one contributes to an understanding of the other. Despite the achievements in the two separate tasks of emotion recognition and depression detection, there has not been much prior effort to build a unified model that can connect these two tasks with different modalities, including multimedia (text, audio, and video) and unobtrusive physiological signals (e.g., electroencephalography). We propose a novel temporal convolutional transformer with knowledge embedding to address the joint task of depression detection and emotion recognition. This approach not only learns multimodal embeddings across domains via the temporal convolutional transformer but also exploits special-domain knowledge from medical knowledge graphs to improve the performance of detection and recognition. It is essential that the features learned by our method can be perceived as a priori and are suitable for increasing the performance of other related tasks. Our method illustrates the case of “two birds with one stone” in the sense that two or more tasks can be efficiently handled with our unique model, which captures effective features. Experimental results on ten real-world datasets show that the proposed approach significantly outperforms other state-of-the-art approaches. On the other hand, experiments in which our methodology is applied to other reasoning tasks show that our approach effectively supports model reasoning related to emotion and improves its performance.