An Explainable AI-Enabled Clinical Decision-Support Framework for Urticaria Identification Using Structured Anamnesis Data: A Pilot Study

Although artificial intelligence has shown promising results in medical diagnosis, limited attention has been devoted to explainable frameworks for clinical decision support using structured anamnesis data whose potential clinical applications require prospective validation and workflow evaluation. In this retrospective proof-of-concept study, we propose an explainable AI framework for urticaria identification integrating machine learning models with post-hoc explainability techniques. The retrospective analytical cohort comprised 99 symptomatic adults from two participating primary-care group-practice settings in the province of Bari, Italy, including 50 patients with a final specialist-established diagnosis of urticaria and 49 symptomatic patients in whom urticaria was not confirmed. Gradient Boosting, Random Forest, and Logistic Regression were evaluated using repeated nested stratified cross-validation with ten repetitions, a five-fold outer loop for performance estimation, and a five-fold inner loop for hyperparameter optimization. All data-dependent preprocessing and hyperparameter tuning were performed exclusively within the corresponding training partitions. Logistic Regression achieved the highest mean AUC-ROC across repetitions (0.803 ± 0.024), whereas Random Forest achieved the highest mean recall (0.886 ± 0.016). The reported predictive performance reflects internal validation within the same retrospective analytical cohort. No independent external or temporal validation was performed. Explainability analyses provided insight into the clinical variables influencing model predictions. The explainability analyses characterize model-derived feature contributions but do not demonstrate improvements in clinician trust, patient understanding, shared decision-making, or clinical workflow outcomes. These potential applications require prospective evaluation involving clinicians and patients.