Identifying depressive users in Twitter using multimodal analysis

Recently, many efforts have been spent on observing individual's psychological states through analyzing users' social activities on SNS. In this paper, we propose a novel method for identifying the users with depressive moods by analyzing their daily tweets for a long period of time. Then, for more accurately understand their tweets, we exploit all media types of tweets, i.e., images and emoticons as well as texts, thus develop a multimodal method for analyzing them. In the proposed method, three single-modal analyses are first performed for extract the hidden users' moods from text, emoticon, and images: a learning based text analysis, a word-based emoticon analysis, and a SVM based image classifier. Thereafter, the extracted moods from the respective analyses are integrated into a mood and again aggregated per a day, which allows for continuous monitoring of user's mood trends. To assess the validity of the proposed method, two types of experiments were performed: 1) the proposed multimodal analysis was tested with a number of tweets, and its performance was compared to SentiStrength; 2) it was applied to classify 45 users' mental states as depressive and non-depressive ones. Then, the results demonstrated that the proposed method outperforms the baseline, and it is effective in finding depressive moods for users.