Abstract Background Artificial intelligence (AI) chatbots may be an asset to patient-provider communication, but not enough is known about how patients respond and how chatbots answer patients’ questions. Objective How perceptions of empathy, quality, trust, liking, and goodness vary by both the actual and perceived source of responses to patient questions (chatbot vs. actual physician). We also coded and compared key verbal elements in chatbot and physician responses. Design This cross-sectional experimental study used chatbot and physician responses from Ayers et al. (2023) in a 2 (actual source: chatbot vs. physician) × 2 (perceived source: chatbot vs. physician) factorial design. Participants U.S.-based, English-speaking participants were recruited online ( N = 1454). Main Measures Participants rated responses on empathy, quality, trust, liking, and goodness. Verbal content of the chatbot and physician responses was independently coded by trained research assistants to identify elements contributing to higher empathy ratings by participants. Key Results Replicating Ayers et al. (2023), participants rated chatbot responses as more empathic than physician responses (Cohen’s d = 0.56, p < 0.001). Chatbot responses received higher empathy ratings than physician responses regardless of what participants were told about authorship (η p 2 = 0.60, p < 0.001). Empathy ratings were higher when participants thought the response was physician-authored, whether it was or not (η p 2 = 0.17, p < 0.001). Participant ratings of quality, trust, liking, and goodness followed the same pattern as empathy. Chatbot responses contained more coder-rated validation, reassurance, and non-judgmental language and were less rushed and more structured than physician responses (Cohen’s d = 0.32 to 1.82, p’s < 0.01). Conclusions AI-generated responses, with human oversight, could enhance computer-mediated clinical communication, although patient awareness of AI contributions may reduce perceptions of empathy. Identification of the specific verbal elements in AI-generated responses could augment communication and increase perceptions of empathic care.
Ruben et al. (2025) studied this question.