A Comprehensive Evaluation of the Performance of Large Language Models on the Japanese National Examination for Radiological Technologists

Key Points

• To evaluate the response accuracy of multiple large language models on the national examination for radiological technologists.
• Analyzed 12 years of national examination questions, excluding inappropriate items.
• Five large language models were tasked with answering the exam questions.
• Calculated response accuracy for all questions and categorized by problem type.
• Gemini 2.5 Pro demonstrated the highest overall response accuracy among the models.
• Gemini 2.5 Pro consistently showed high accuracy across different problem types.
• Other LLMs exhibited varying accuracy trends depending on the problem format.