Utility of large language models in pharmacology examinations: Comparative evaluation of ChatGPT, Gemini, and Perplexity

BACKGROUND: Artificial intelligence (AI) tools, including large language models (LLMs) including ChatGPT, Gemini, and Perplexity, are being increasingly incorporated into medical education. Their ability to answer higher-order pharmacology questions, however, remains underexplored. OBJECTIVE: The objective of this study was to evaluate the effectiveness of three AI platforms – ChatGPT, Gemini, and Perplexity – in answering university-level pharmacology examination questions. MATERIALS AND METHODS: Three recent university pharmacology university examination papers were selected, including multiple-choice questions (MCQs), case-based questions, short notes, brief answers, and pharmacotherapy essay questions. Independent responses were generated using ChatGPT, Gemini, and Perplexity. A pharmacology faculty member assessed the generated responses using a structured scoring rubric. Quantitative analysis was done using the Kruskal–Wallis test to compare mean rank scores, whereas qualitative analysis assessed the responses based on presentation, clarity, accuracy, length, and relevance. RESULTS: While Gemini achieved higher mean scores in case-based (33/42) and short-note (56.5/70) formats, ChatGPT performed better in pharmacotherapy questions (10/18) and MCQs (60/60). Perplexity generated concise answers but achieved lower scores across all categories of questions. No statistically significant differences were observed among the three platforms ( P > 0.05). Qualitative analysis showed that Gemini’s responses were well-structured and moderately detailed, ChatGPT’s responses were accurate but sometimes overly verbose, whereas Perplexity’s were concise but lacked explanatory depth. CONCLUSION: All three platforms were found to be useful in answering pharmacology questions, although each showed distinct strengths. Gemini excelled in producing balanced and well-structured responses, ChatGPT in detailed analytical reasoning, and Perplexity in concise answer. These findings suggest that LLMs have the potential to enhance pharmacology education; however, their outputs should be critically assessed to improve their utility.