Abstract Background: As artificial intelligence continues to reshape healthcare landscapes, large language models (LLMs) have emerged as powerful tools with potential applications spanning clinical documentation, patient communication, and diagnostic support. However, the clinical utility and safety profiles of these models remain inadequately characterized. This study evaluates four prominent LLMs across multiple healthcare-relevant domains to provide evidence-based guidance for clinical implementation. Methodology: We developed a comprehensive evaluation framework assessing ChatGPT, Google Gemini, Perplexity, and Grok versions as of JULY 2025 across 15 clinical domains. Our methodology included standardized testing scenarios designed to mirror real-world healthcare applications, from emergency medicine protocols to evidence-based research synthesis. Performance metrics encompassed accuracy, speed, safety, and integration capabilities. Results: Our analysis revealed significant performance variations among models. ChatGPT achieved the highest composite score (82/100), demonstrating particular strength in clinical documentation and patient communication. However, critical safety concerns emerged across all platforms, including dangerous product hallucination rates (23%–31% in Google Gemini and Grok) and universal patient memory limitations. Perplexity distinguished itself as the only model providing consistent source citations (94% accuracy), while Grok’s rapid response times (average 12.3 s compared to ChatGPT 28.7 S, Gemini 31.2 S, and perplexity 24.8 s) showed promise for emergency applications despite translation accuracy concerns. Conclusions: While LLMs demonstrate substantial potential for enhancing clinical workflows, current models require careful implementation with robust safety protocols. No single model provides comprehensive clinical utility, suggesting multi-platform strategies may be necessary. Critical safety gaps, particularly in medical product recommendations and patient data continuity, demand immediate attention before widespread clinical deployment.
Hirani et al. (Wed,) studied this question.