Consistency over accuracy: run-to-run stability of contemporary large language models on Turkish curriculum-aligned theoretical anatomy multiple-choice questions

Key Points

• This research aims to evaluate the accuracy and stability of large language models on anatomy multiple-choice questions in Turkish.
• Analyzed responses of contemporary large language models on curriculum-aligned anatomy questions.
• Evaluated single-trial accuracy versus run-to-run stability metrics.
• Assessed consistent-correct and consistent-wrong rates for stability appraisal.
• Models demonstrated high accuracy on individual trials but exhibited substantial volatility.
• Run-to-run stability varied, indicating systematic errors not visible in single trials.
• Recommendations include prioritizing stability in the selection of models for educational use.