Development and Validation of Algorithms for Systemic Sclerosis Identification in Electronic Health Record Data

Objective The aim of this study was to develop and validate International Classification of Diseases (ICD) code–based algorithms for identifying systemic sclerosis (SSc) cases within electronic health record (EHR) data and to evaluate algorithm performance. Methods We identified patients with at least one ICD, Ninth Revision (ICD‐9)/ICD‐10 code for SSc in a large multicenter EHR dataset (TriNetX Research Network). A random sample of patients underwent detailed medical record review to confirm SSc diagnosis (gold standard). All ICD‐9/10 codes assigned during inpatient or outpatient encounters and corresponding dates were extracted. SSc case status derived from seven prespecified algorithms was compared with the gold standard by calculating sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and receiver operating characteristic analyses. Results Medical records from 549 of 1,098 patients with at least one SSc ICD‐9/10 code were reviewed; 399 (72.7%) were confirmed SSc cases. The algorithm requiring at least two outpatient ICD‐9/10 SSc codes at least 30 days apart with exclusion of scleroderma mimics within 24 months demonstrated the best overall performance (sensitivity 96%, specificity 71%, PPV 90%, NPV 87%, and area under the curve 0.91). Algorithms based on at least one inpatient ICD‐9/10 code showed high specificity (91%) and PPV (93%) but poor sensitivity (45%) and NPV (38%). Adding inpatient codes to the outpatient code–based algorithms did not improve performance. Conclusion An algorithm requiring at least two outpatient SSc ICD‐9/10 codes ≥30 days apart without scleroderma mimics within 24 months reliably identifies SSc cases. This readily implementable approach outperforms previously published more complex algorithms and supports valid clinical and epidemiologic research in large datasets.