Fake News Detection with Large Language Models on the LIAR Dataset

Abstract The widespread dissemination of fake news poses a significant threat to the integrity of information. Detecting fake news with high accuracy is crucial for maintaining the integrity of information in the digital age. The evaluation of ChatGPT and Google Gemini models for this task has revealed their substantial capabilities in discerning the veracity of statements, highlighting their potential to mitigate the spread of misinformation. Using the LIAR benchmark dataset, the study demonstrated high performance metrics across accuracy, precision, recall, F1 score, and AUC-ROC, emphasizing the effectiveness of these models in real-world applications. The comparative analysis and error examination provided insights into the strengths and limitations of each model, offering valuable guidance for future enhancements. Practical implications include the integration of these models into fact-checking systems to improve content verification processes, supporting media organizations and social platforms in their efforts to combat misinformation. The findings prove the importance of ongoing research and development to refine and optimize LLMs, ensuring their continued relevance and efficacy in addressing the challenges posed by fake news.