Accuracy of AI-generated procedural descriptions in peripheral nerve surgery: implications for surgical training

Purpose Self-directed learning is increasingly important in surgical education. Large language models may support this, however, their accuracy, reliability, and sensitivity to model generation and prompting strategy remain unclear. This study evaluated AI-generated procedural descriptions in peripheral nerve surgery using GPT-5.2, and tested robustness using a newer model, GPT-5.5. Methods Fifteen peripheral nerve procedures were randomly selected from the Washington University in St. Louis Surgical Education Website as reference standards and coded into steps. GPT-5.2 outputs were evaluated for accuracy and other response characteristics using a structured prompt. Regression assessed accuracy against complexity, word count, steps, and step density. For five procedures, GPT-5.5 was evaluated using structured and simple prompts. Results Mean GPT-5.2 accuracy was 81.2% (SD 11.5%) across 90 responses, ranging from 68.5% to 91.7%. Accuracy was positively associated with word count and negatively associated with step density, though neither remained independent predictors. Of 195 missed steps, most involved judgement-based maneuvers (86/195), major steps (56/195), or neurovascular identification (41/195). GPT-5.5 accuracy (81.4%, SD 16.2%) resembled GPT-5.2’s but was more variable, decreasing to 40.5% (SD 40.2%) under a simple prompt, shifting omissions towards major steps. Conclusion GPT-5.2 and GPT-5.5 generated accurate but variable responses, inconsistently capturing judgement-dependent content. Simple prompting substantially reduced accuracy and shifted omissions toward major steps, indicating dependence on prompting strategy. AI-generated descriptions may aid orientation, but should not replace expert guidance for safety-critical and judgement-required steps.