Generative AI in the Taiwan medical laboratory scientist examination: Performance and educational utility

BACKGROUND: The rapid evolution of generative artificial intelligence (GAI), particularly large language models (LLMs), offers transformative potential for health professions education. However, their performance in specialized clinical domains, such as medical laboratory science as well as educational utility, remain insufficiently characterized. This study evaluated multiple LLMs on the Taiwanese Medical Laboratory Scientist Examination (TMLSE) and investigated the effectiveness of AI-generated assessments in undergraduate-year (UGY) medical laboratory training. METHODS: Four LLMs-GPT-3.5, GPT-4o, GPT-5.2, and Gemini 3 Pro-were evaluated using six TMLSE administrations from 2021 to 2023. Accuracy was assessed across 10 disciplines and image-based items, with incorrect responses categorized into informational, logical, or statistical errors. Additionally, 180 AI-generated items were validated by experts and administered to 30 UGY students. Student satisfaction and correlations between assessment performance and clinical internship outcomes were analyzed. RESULTS: All the models surpassed the 60% passing threshold, with clear generational improvement. GPT-5.2 (93.5%) and Gemini 3 Pro (92.1%) significantly outperformed GPT-4o (85.8%) and GPT-3.5 (67.1%). An error analysis indicated that informational deficiencies were the predominant failure mode, accounting for more than 95% of the incorrect responses across all the models. Although image-based items were more challenging than text-based questions were, GPT-5.2 demonstrated superior multimodal capability, achieving an image-based accuracy of 73.3%. In the educational evaluation, UGY students rated AI-generated items significantly higher than authentic TMLSE questions did in terms of overall satisfaction (p= 0.0089) and perceived utility for examination review (p= 0.0405). The performance on AI-generated assessments was positively correlated with clinical internship outcomes (Spearman's r= 0.3662, p= 0.0465), whereas no significant association was observed with traditional TMLSE scores. CONCLUSION: Modern LLMs demonstrate high performance in specialized medical laboratory knowledge, although image interpretation remains a limitation. Integrating AI-generated assessments into UGY curricula represents an effective and scalable approach to competency-based education, with demonstrated alignment to real-world clinical performance.