Comparison of the performance of large language models (ChatGPT-5.5, Gemini 3.1, and Claude 4.6) in answering pharmacology questions

Large language models (LLMs) have introduced a fundamental transformation in healthcare and medical education. This study aimed to evaluate the performance of LLMs in answering pharmacology questions. This quantitative cross-sectional study was conducted in Iran in 2026. A 50-item Persian multiple-choice questionnaire assessing pharmacology knowledge was administered to ChatGPT-5.5, Gemini 3.1, and Claude 4.6. Data were analyzed using confusion matrix methods, Cochran’s Q, McNemar, and point-biserial tests. Sensitivity values were 1.00 for ChatGPT-5.5, 0.94 for Gemini 3.1, and 0.90 for Claude 4.6. Specificity values were 0.43, 0.50, and 0.35, respectively. Accuracy was 0.82 for both ChatGPT-5.5 and Gemini 3.1, and 0.72 for Claude 4.6. Overall, the mean sensitivity, specificity, and accuracy across all LLMs were 0.95, 0.43, and 0.78, respectively. The analysis further revealed a statistically significant decline in performance of ChatGPT-5.5 and Gemini 3.1 when addressing quantitative questions (φ = −0.286, p < .05). LLMs have potential to support professionals in educational/knowledge contexts of pharmacology, provided ethical considerations are addressed, though further research is needed to better understand their performance and limitations.