XLNet-LSTM-CNN for Text Sentiment analysis

Abstract Given the burgeoning popularity of social media and online commenting platforms, the significance of text sentiment analysis has been exponentially amplified. Deep learning is being extensively employed for text sentiment analysis, with CNN(Convolutional Neural Network) and LSTM(Long Short-Term Memory Network) showcasing remarkable capabilities. CNNs adeptly extract localized features of text, while LSTMs retain information across longer textual spans. These are often synergistically used for text sentiment classification tasks in tandem with static word vector techniques, for instance, word2vec. We propose a novel text sentiment classification model, designated as XLNet-LSTM-CNN, amalgamating the strengths of CNN and LSTM with the XLNet model. This hybrid model addresses the constraints of static word vector techniques such as word2vec, including their inability to glean deep information about the text or discern the polysemy of words. We further bolster the robustness of our model through data augmentation methods. We evaluated the performance of our proposed XLNet-LSTM-CNN model using three distinct English movie review datasets (IMDB, MR, and SST), comparing it with seven other prevalent neural network models(SVM, CNN, RNN, LSTM, BiLSTM, CNN-LSTM, and CNN-attention-LSTM). Our experimental findings revealed that the XLNet-LSTM-CNN model not only attains superior accuracy but also minimizes the loss rate and augments the model's generalization capacity compared to the competitor models. In summation, our proposed model provides a robust solution for text sentiment classification, holding immense potential for wide-scale application in text sentiment analysis on social media and online commenting platforms.