Interpretation Checkpoint: LLM-Assisted Educationfor Detecting Deceptive Interpretations of Visual Data

Data visualizations, even if designed correctly, can still be deceptive through a textual narrative context that may accompany them, posing a challenge to data literacy. This study introduces a prompt-conditioned LLM-based reasoning module designed to identify deceptive narratives and a PlotTwist DataSet (PTDS), the first-ever curated dataset to support research on deceptive interpretations of data visualizations. In a randomized, controlled experiment with university students (N=192), we compared LLM-supported instruction to traditional classroom and e-learning methods. Pre-intervention tests revealed susceptibility to deceptive narratives, and post-intervention results showed improvement across all methods, with LLM-based instruction yielding gains comparable to traditional classroom and e-learning approaches, with no statistically significant differences between groups. Additionally, we compare multiple LLMs’ performance to a supervised MatCha→BERT pipeline performing multi-class classification, achieving a Macro F1-score of around 0.89 on the PTDS dataset. Results show that leading LLMs match or slightly exceed this baseline, although the performance gains from incorporating visual information are modest. These findings demonstrate the effectiveness of LLM-supported education in enhancing participants’ vigilance and reducing vulnerability to disinformation.