Multi-Modal Depression Detection and Estimation

Depression and anxiety disorders are critical problems in modern society. The WHO studies suggest that roughly 12. 8 percent of the world's population are suffering from a depressive disorder. In this work, we propose several novel approaches towards multi-modal depression detection and estimation. Our previous studies mainly explored the multi-modal features and multi-modal fusion strategies, experimental results showed that the proposed hybrid depression classification and estimation multi-modal fusion framework obtains promising performance. The current work contains two parts: 1) In order to mitigate the impact of lack of data on training depression deep models, we utilize Generative Adversarial Network (GAN) to augment depression audio features, so as to improve depression severity estimation performance. 2) We propose a novel FACS3D-Net to integrate 3D and 2D convolution network for facial Action Unit (AU) detection. As far as we know, this is the first work to apply 3D CNN to the problem of AU detection. Our future work will 1) focus on combining depression estimation with dimensional affective analysis through the proposed FACS3D-Net, and 2) collect Chinese depression database. When completed, these studies will compose the author's dissertation.