Answers are grounded in this paper and cite their sources.
Background Patients and caregivers increasingly rely on LLMs for health information, yet their performance in TT, a time-critical urological emergency, remains insufficiently characterized. This study compared three LLMs across content safety, quality, reliability, readability, understandability, and actionability. Methods Twenty-five standardized patient-centered questions were developed based on clinical guidelines, expert consensus documents, and information needs identified through Google Trends. Each question was submitted to GPT-5.6 Sol (High reasoning mode), DeepSeek-V4-Pro, and Doubao 2.0 Expert Model, yielding 75 responses. Quality and reliability were assessed using GQS, mDISCERN, and QAMAI; understandability and actionability were evaluated using PEMAT-U and PEMAT-A, respectively. Readability was assessed using six English-language indices, and content deficiencies were also evaluated. Results GPT-5.6 Sol(High reasoning mode) achieved significantly higher GQS, mDISCERN, and QAMAI scores than the other two models (all overall P < 0.001). Hallucinations and factual inaccuracies occurred in 28.0% and 17.3% of responses, respectively. No hallucinations were detected in GPT-5.6 Sol responses, compared with 44.0% for DeepSeek-V4-Pro and 40.0% for Doubao 2.0 Expert Model. Understandability did not differ significantly among models ( P = 0.0735), whereas actionability did ( P < 0.001), with GPT-5.6 Sol(High reasoning mode) performing best. All six readability indices differed significantly among models (all P < 0.001). DeepSeek-V4-Pro showed the most favorable relative readability, GPT-5.6 Sol(High reasoning mode) intermediate performance, and Doubao 2.0 Expert Model the greatest reading difficulty. Conclusions The three LLMs exhibited distinct performance profiles in generating TT-related health information. GPT-5.6 Sol (High reasoning mode) achieved the highest scores for quality, reliability, and actionability, whereas DeepSeek-V4-Pro showed comparatively better readability. Understandability did not differ significantly among models. However, hallucinations, factual inaccuracies, and substantial reading demands remain concerns. These findings reflect model performance under the specific evaluation conditions and at the time of data collection, rather than established clinical effectiveness or safety. GPT-5.6 Sol(High reasoning mode) may be the most suitable supplementary information tool among the models evaluated, but it should not replace professional medical assessment or clinical decision-making.
No takes yet. Share an insight, caveat, or question.
Dai et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: