When an AI Says It Is False: User Responses to Misinformation Flagging by Automated vs. Human Fact-Checkers

To combat misinformation at scale, automated fact-checkers are being deployed, but we do not know if lay users trust them. Are AI fact-checkers trusted more than human fact-checkers because of their accuracy in identifying tell-tale features of fake news? Or are they trusted less because they are seen as lacking the subjectivity necessary for corroborating evidence? A pre-registered 2 (Fact-checking source: Human vs. AI) × 3 (Fact-checking approach: Evidence-based vs. Feature-based vs. Black-box) between-subjects experiment among 291 US adults recruited from Cloud Research revealed that users’ trust was predicted by the extent to which the interface triggered the positive machine heuristic (the algorithm is more objective and precise than human) and the negative machine heuristic (the algorithm lacks human subjective judgment). The latter was more likely when the system used an evidence-based determination of misinformation, which was better understood by users than a feature-based approach. Theoretical and practical implications for individuals’ trust of automated fact-checkers are discussed.