Comparison of large language models in answering early gastric cancer clinical problems and recognizing multimodal endoscopic images

Large language models (LLMs) show clinical potential. We evaluated the performance of LLMs, including ChatGPT, DeepSeek, Gemini, and Grok, in solving early gastric cancer (EGC) clinical problems and analyzing multimodal endoscopic images. For the text evaluation, we developed 40 EGC clinical questions and 50 single-choice questions (SCQs) based on guidelines. Physicians and patients rated the responses generated by different LLMs according to accuracy, relevance, completeness, and comprehension, and assessed their readability. In the image evaluation, we collected a total of 482 white-light endoscopy (WLE) images and 478 narrow-band imaging (NBI) images from four hospitals in China to explore the ability of different LLMs to recognize EGC in static endoscopic images. ChatGPT-4o obtained the highest scores for accuracy (86, P < 0.05), relevance (88, P < 0.05), and comprehension (88, P < 0.05), whereas Grok-3 demonstrated superior completeness (90, P < 0.05). DeepSeek-V3.2 achieved the highest SCQ accuracy (88%, P = 0.268). In terms of pathological type recognition, the sensitivity of ChatGPT-5.2, Gemini-3.0 pro, and Grok-4 in WLE images were 65.08%, 99.60%, and 81.75%, respectively, and the specificity was 47.92%, 0%, and 29.17%, respectively; ChatGPT-5.2, Gemini-3.0 pro, and Grok-4 in WLE images were 57.54%, 99.60%, and 92.86%, respectively, and the specificity was 60.42%, 0%, and 2.08%, respectively. LLMs have shown potential in addressing clinical questions related to EGC, but their ability to recognize EGC in static multimodal endoscopy is limited and their effectiveness is inconsistent.