Multimodal Sentiment Analysis via RNN variants

Multimodal sentiment analysis involves the classification of sentiment by using different forms of data together, namely, text, audio, and video. Previously sentiment analysis was implemented only on textual data. Multimodal sentiment analysis relies mainly on identifying the utterances present in the video and using them as a basis for sentiment classification. In this paper, we proposed four different variants of RNN, namely, GRNN, LRNN, GLRNN and UGRNN for analyzing the utterances of the speakers from the videos. Experimental results on CMI-MOSI dataset demonstrates that our approach is able to achieve better sentiment classification accuracy on individual modality (text, audio, and video) than existing approaches on the same dataset. Moreover, our method also gave decent results after fusing the individual modality using Attention networks for multimodal sentiment analysis.