Fake News Detection on Social Media using Logistic Regression and Compare Textual Property with Decision Tree Algorithm

The study used machine learning algorithms LR and DT which were assessed via 10 fold cross validation on a combined set of 44,898 labeled news articles (23,481 fake, 21,417 real) from the WELFake Kaggle dataset. We applied TF-IDF vectorization which included unigram features, removed English stop words and used max_features=50,000 for feature extraction. We report that LR achieved a mean accuracy of 96.52% ±0.41%, precision of 96.10%, recall of 95.60% and F1 score of 95.85% which in turn outperformed DT which reported 88.34% ± 1.12% accuracy. Also we evaluated a Naive Bayes baseline which reported 90.21%. The difference between LR and DT is statistically significant ($p<0.001$, Independent Sample T-Test, IBM SPSS).