Answers are grounded in this paper and cite their sources.
Background Large language model (LLM)-based chatbots are increasingly used for health information, but their suitability for melanoma public education requires evaluation across accuracy, information quality, transparency, and readability. Methods We compared ChatGPT-5.6, DeepSeek-V4, Claude Sonnet 5, and Gemini 3.5 Flash. Validity was assessed using 200 melanoma-related single-best-answer multiple-choice questions administered in three independent trials. Fifteen core public-education questions were developed from worldwide Google Trends data and clinical experience. Responses were evaluated using DISCERN, EQIP, GQS, and JAMA benchmarks, and readability was assessed using ARI, Coleman–Liau, FKGL, FRES, GFI, and SMOG indices. Results Mean MCQ accuracy was highest for ChatGPT-5.6 (94.0% ± 0.5%), followed by Gemini 3.5 Flash (92.5% ± 1.3%), Claude Sonnet 5 (87.2% ± 1.4%), and DeepSeek-V4 (79.3% ± 0.3%; p < 0.001). ChatGPT-5.6 achieved the highest DISCERN (52.07 ± 9.48), EQIP (81.67 ± 4.08), and GQS (4.40 ± 0.51) scores. JAMA transparency scores were low across models. Gemini 3.5 Flash showed the most favorable overall readability profile; however, mean grade-level indices for all models exceeded the sixth-grade benchmark, and all mean FRES scores were below the recommended range. Conclusion Current LLM chatbots show substantial potential for melanoma education, but performance varies across validity, quality, transparency, and readability. Their use should remain supplementary, with improved source transparency and health-literacy–appropriate language required before broader patient-facing implementation.
No takes yet. Share an insight, caveat, or question.
Wei et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: