To Trust or Distrust AI: A Questionnaire Validation Study

Despite the importance of trust in human-AI interactions, researchers must often rely on questionnaires adapted from other fields, which lack validation in the AI context.Motivated by the need for reliable and valid measures, we investigated the psychometric quality of the most commonly used trust questionnaire in the context of AI by Jian, Bisantz, and Drury (2000).In a pre-registered online experiment ( = 1485), participants observed interactions with both trustworthy and untrustworthy AI and rated their trust.Our results did not support the originally proposed single-factor structure for the questionnaire, but instead suggested a two-factor solution that distinguishes between trust and distrust.Based on our findings, we provide recommendations for future studies on how to use the questionnaire.Finally, we present arguments for considering trust and distrust as two distinct constructs, emphasizing the opportunities of considering and measuring both in human-AI interactions.