Methods of improving sentiment analysis based on sentiment dictionary

Sentiment analysis relying on predefined dictionaries often suffers from reduced accuracy due to biases in the sentiment scores assigned to certain words. To overcome this limitation, we introduce a novel framework that integrates a sentiment dictionary with a Naive Bayes machine learning approach for sentiment analysis. This framework seamlessly combines dictionary creation and sentiment classification into a unified process. To improve the quality of sentiment predictions, we propose two complementary techniques. The first technique employs statistical analysis to detect and eliminate potential stopwords that may skew results, while the second imposes a symmetric distribution on positive and negative sentiment scores by adjusting their mean and variance. We constructed multiple sentiment dictionaries using various tokenizers and datasets to assess the proposed methods. Experimental results demonstrate that these techniques successfully enhance sentiment analysis performance by reducing biases inherent in the sentiment dictionary.