Evolution of guideline-based male infertility responses by large language models between 2025 and 2026: a comparative analysis of ChatGPT, Perplexity, and DeepSeek

Background Large language models (LLMs) are increasingly used by patients and clinicians to obtain medical information, yet their performance may change over time as models are updated. Male infertility is a guideline-sensitive field in which inaccurate or incomplete information may affect diagnostic evaluation, counseling, and treatment expectations. This study evaluated the year-on-year evolution of ChatGPT, Perplexity, and DeepSeek in responding to guideline-derived male infertility questions between 2025 and 2026. Results A total of 528 chatbot responses were evaluated from 88 European Association of Urology (EAU) guideline-derived male infertility questions answered by ChatGPT, Perplexity, and DeepSeek in 2025 and 2026. In 2025, significant inter-model differences were observed across all four scored domains, with Perplexity achieving the highest content-quality and expert-rated scores and DeepSeek showing the weakest performance. In 2026, DISCERN scores converged across models, whereas significant differences persisted for readability, Global Quality Score, and expert opinion. ChatGPT showed significant improvement across all four scored domains and achieved the highest Global Quality Score in 2026. DeepSeek demonstrated the most pronounced improvement, particularly in DISCERN and expert opinion scores. Perplexity improved in readability but did not show a parallel positive change in Global Quality Score or expert-rated clinical usefulness. Potential patient-safety concerns were identified in seven of the 88 DeepSeek responses from 2025, whereas no such concern was identified in the 2026 responses. None of the 528 responses contained an explicit recommendation considered likely to cause immediate or serious patient harm. Conclusions Performance changed in a model-specific rather than uniformly progressive manner. The findings represent sampled first-pass responses to clearly formulated guideline-derived questions rather than stable estimates of model performance or reproducibility. LLMs may support access to male infertility information, but verification against authoritative sources and appropriate human oversight remain necessary.