Health Literacy, Chatbot Use, and the Patient-Physician Perception Gap After Previsit Artificial Intelligence-Generated Document Summaries: Secondary Analysis of a Randomized Controlled Trial (Preprint)

BACKGROUND Patients often cannot understand their own medical documents. In the AI-INFOCARE randomized controlled trial, a quality-checked, artificial intelligence (AI)-generated lay-language summary of these documents, provided before a first neurosurgical consultation, improved patient-rated interaction quality and autonomy support and shortened consultations, whereas physicians rated their own autonomy support slightly lower. Whether these benefits depend on health literacy or prior information seeking, including large language model (LLM) chatbot use, and how closely patients and physicians agree on the same encounter, is unknown. OBJECTIVE We aimed to describe previsit information seeking, document comprehension, and health literacy among neurosurgical outpatients; to quantify patient-physician agreement on autonomy support; and to estimate whether intervention effects vary by health literacy, information seeking, and age. METHODS In this single-center, open-label trial in a German neurosurgical outpatient clinic, 300 adults were randomized 1:1 to usual care with or without the summary. A previsit questionnaire (information sources, document comprehension, 3 Chew-based health literacy screening items) was added on July 24, 2025, and completed by 174 of 181 participants enrolled thereafter. Outcomes were patient-rated and physician-rated autonomy support on the Health Care Climate Questionnaire (HCCQ), patient-rated interaction quality on the Questionnaire on the Quality of Physician-Patient Interaction (FAPI), and consultation time. Agreement was quantified with correlations, Bland-Altman limits, and physician-level intraclass correlation coefficients, and effect modification with linear mixed models (physician random intercepts), reporting P values for 3 prespecified contrasts only. RESULTS Of 300 participants (mean age 57.2, SD 15.6 years; 153/297, 51.5% female), 139/174 questionnaire respondents (79.9%) had sought information before the visit, 26 (14.9%) from an LLM chatbot (14/50, 28% of those aged <50 years vs 11/121, 9.1% of older participants). Only 40/174 (23.0%) understood their documents well, and 99 (56.9%) screened positive for limited health literacy. Patient and physician HCCQ ratings correlated weakly (r=0.17, 95% CI 0.05-0.28); physicians accounted for 59% (95% CI 32%-82%) of the variance in their self-ratings vs 16% (95% CI 2%-25%) in patient ratings. Under usual care, physicians rated their autonomy support higher than their patients did (mean difference −0.17, 95% CI −0.25 to −0.09), which the intervention reversed (0.09, 95% CI 0.01-0.18). Intervention effects on patient-rated HCCQ and FAPI did not differ by health literacy (P=.90 and P=.37 for interaction) or prior information seeking (P=.15), and age did not modify physician self-ratings once physician clustering was modeled (P=.86; P=.01 in a cluster-robust analysis without physician effects). CONCLUSIONS AI-generated summaries benefited patients regardless of health literacy and prior information, including LLM chatbot users. Patient and physician perceptions diverged, and physician self-ratings reflected the physician more than the encounter; physician-reported outcomes should be analyzed with physician clustering and not serve as sole outcomes in communication trials. CLINICALTRIAL Trial Registration: Deutsches Register Klinischer Studien DRKS00036810; https://drks.de/search/en/trial/DRKS00036810 International Registered Report Identifier (IRRID): RR2-10.2196/77204