Characteristics of Datasets for Fake News Detection to Mitigate Domain Bias

"Fake news", news intentionally containing false information, has become quite common and often causes social disruption. Many researches on automatic detection of fake have been extensively studied. The classification accuracy is improving, but a major challenge for practical application still remains: models can not work well for news in unknown fields, called "domains", due to bias caused by different words and phrases among domains. To improve the accuracy of cross-domain fake news detection, it is crucial to mitigate the domain bias since unknown news articles to be classified can be in unknown domains. As a preliminary experiment, we trained a classifier using news articles whose noun phrases were masked because they are considered as a major source of the bias. However, contrary to expectations, masking did not improve accuracy. From the preliminary experiment, we obtained the hypothesis that pairs of fake and real news on the same topic can mitigate the domain bias. Using comparative experiments, we show that accuracy is higher when trained on paired news articles than when trained on unpaired ones.This result strongly suggests that a fake news dataset consisting of paired news could be effective for cross-domain detection.