How well do DeepSeek, ChatGPT, and Gemini respond to water science questions?

This study aims to evaluate the performance of three prominent LLMs, DeepSeek R1, ChatGPT-4o, and Gemini 2, in addressing key questions within four core fields of hydrology and water science: machine learning and optimization, remote sensing, flood modeling, and sediment transport. LLMs’ responses are systematically compared to benchmark responses derived from review articles in the respective fields. To assess the LLMs’ efficiency, a novel evaluation rubric is introduced in this study, incorporating four key criteria: relevancy, accuracy, authenticity, and novelty. Findings revealed that each model can address the core aspects of the benchmark questions. DeepSeek R1 achieved the highest overall scores in machine learning and optimization, flood modeling, and sediment transport, while ChatGPT-4o demonstrated superior performance in remote sensing. Notably, DeepSeek R1 and Gemini 2 exhibited the lowest response similarity in 95% of the evaluated questions, whereas ChatGPT-4o and Gemini 2 showed the highest similarity in 70% of cases. • Developed a novel framework for evaluating LLM responses in water science and hydrology, incorporating a scoring rubric, word matching algorithm, similarity analysis, and generation speed. • DeepSeek R1 achieved the highest overall scores in three out of four topics. • Gemini demonstrated the fastest response generation speed. • LLMs showed the capability to extend beyond benchmark answers by integrating broader scientific knowledge.