Variability in Nurses’ Assessments and Concordance of Multimodal Generative AI Models in Image-Based Wound Assessment

This is the authors' abstract. We don't add key points for this paper.

Background/Objectives: Image-based wound assessment may vary among experienced nurses. This study examined inter-nurse variability and concordance with three multimodal generative artificial intelligence models. Methods: In this exploratory cross-sectional comparative study, 18 nurses with wound-care expertise and ChatGPT, Gemini, and Claude independently assessed 30 de-identified wound images from León, Spain, across seven assessment and treatment-related domains. Inter-nurse agreement was summarized using modal shares and Gwet’s AC1. Concordance with the nurse modal set required an exact match with any response in the nurse modal set. Overall confidence intervals were estimated by case-clustered bootstrap resampling. No independent clinical reference standard was established. Results: The analyses quantified inter-nurse agreement and model concordance with the nurse modal set, not diagnostic accuracy. Inter-nurse agreement was highest for wound etiology (mean modal share, 71.3%; AC1, 0.520) and lowest for primary dressing selection (43.7%; AC1, 0.163). Overall concordance was 63.3% (95% confidence interval, 56.7–70.0) for Gemini, 48.6% (40.5–56.2) for ChatGPT, and 44.8% (38.6–51.4) for Claude. Gemini exceeded ChatGPT by 14.8 percentage points and Claude by 18.6 percentage points in exploratory comparisons of overall concordance with the nurse modal set. Complete intermodel agreement occurred in 38.6% of 210 case–domain tasks. Conclusions: Image-based wound assessments varied among experienced nurses, and model concordance with the nurse modal set differed across assessment domains and evaluated systems. Concordance with the nurse modal set reflects agreement with the panel’s most frequently selected responses. It does not establish clinical correctness, clinical validity, or an ability to resolve clinical ambiguity: image-only assessment, the small purposive sample, and reliance on a nurse-derived comparator limit interpretation. Further validation in clinical contexts and against independent reference standards is needed before these models can support wound-care decisions.