Comparative evaluation of pulmonary aspergillosis information generated by large language model-based chatbots

Chatbots powered by large language models are increasingly used for medical information retrieval, yet pulmonary aspergillosis presents complex diagnostic and therapeutic information needs. This comparative cross-sectional study evaluated the information quality, transparency, and readability of responses generated by four chatbot platforms. On 6 August 2026, 31 search terms derived from five-year worldwide Google Trends data were submitted to Microsoft Copilot, ChatGPT (GPT-5.5), Perplexity, and Gemini 3.6 Flash, yielding 124 first-response texts. Quality and transparency were assessed using JAMA, GQS, EQIP, and DISCERN, and readability using SMOG, Coleman-Liau, FKGL, GFI, FRES, and ARI. Friedman tests accounting for matched queries showed significant platform differences for all four quality and transparency measures. Median DISCERN scores were 43 for Copilot and Perplexity, 36 for ChatGPT, and 33 for Gemini. None of the 124 default responses met the prespecified sixth-grade threshold for any of the six readability measures. Copilot and Perplexity achieved higher scores on several measures of written-information quality and transparency, while responses from all four platforms remained textually complex. Chatbot-generated information should serve as a supplementary resource following professional verification.