Let’s ask the patient: disease prediction based on patients’ symptom descriptions in free text

Objective This study evaluates the value self-reported free text symptom descriptions for supporting diagnostic decisions in osteoarthritis (OA), fibromyalgia (FM), and immune—mediated rheumatic diseases (imRD) using natural language processing (NLP) and machine learning (ML). Methods Free-text descriptions from 8,454 patients were processed using a word—weighting method (TF-IDF vectorization) that reflects how relevant each word is across the dataset, and then classified with support vector machine (SVM) models. OA and FM models were optimized for specificity and the imRD model for sensitivity based on disease context and validated against an independent dataset. Model explainability was explored using SHAP values. Results The SVM models demonstrated moderate diagnostic support potential with AUC-ROC values of 0.68 for OA, 0.75 for FM, and 0.69 for imRD. When optimized for clinical utility, the models achieved high specificity of 0.82 for OA and 0.92 for FM, effectively reducing unnecessary referrals with misdiagnosis rates of only 17% and 8%, respectively. For imRD, the model achieved a sensitivity of 0.92 and NPV of 0.77, ensuring minimal missed diagnoses of these potentially serious conditions. Decision curve analysis confirmed clinical utility across varying threshold preferences. SHAP (SHapley Additive exPlanations) analysis revealed that key linguistic patterns in patient descriptions aligned with clinical reasoning, enhancing the models’ interpretability. Conclusion Our results highlight the value of patient-reported data in augmenting rheumatology decision-making and sets the stage for further development in AI-assisted diagnostics. While not a standalone diagnostic tool, the integration of NLP-driven analysis of free-text symptom descriptions shows promise in reducing diagnostic ambiguity.