Evaluating the Readability, Quality, and Reliability of Responses Generated by ChatGPT, Gemini, and Perplexity to the Most Frequently Searched Queries About Acute Mesenteric Ischemia

Aim: Patients increasingly consult artificial intelligence (AI) chatbots before deciding whether to seek emergency care.This study compared the readability, reliability, and quality of responses generated by ChatGPT, Gemini, and Perplexity in response to frequently searched queries about acute mesenteric ischemia. Materials and Methods:Sixteen queries identified through Google Trends were submitted to each chatbot, yielding 48 responses.Readability was assessed with seven formulas.Two evaluators independently rated reliability using the JAMA benchmarks and modified DISCERN and quality using the Global Quality score and the Ensuring Quality Information for Patients tool.Platforms were compared with the Friedman and Wilcoxon signed-rank tests.Results: ChatGPT produced shorter responses than Gemini and Perplexity (medians 180.5, 390.0, and 416.5 words; p<0.001).All platforms' content was written at a college reading level (median Flesch-Kincaid Grade Level of 14.0-15.5),well above the recommended sixth to eighthgrade level.Perplexity obtained the highest JAMA and Ensuring Quality Information for Patients scores, whereas Gemini and Perplexity outperformed ChatGPT on the modified DISCERN and the Global Quality score. Conclusion:No chatbot was superior across all measures, and none produced text readable by the average patient.Information on this timecritical emergency should be simplified and professionally reviewed before being recommended to patients.The clinical accuracy and the appropriateness of emergency referral advice were not assessed and require further evaluation.