Development and validation of a text-mining-based algorithm for detection of rare hereditary periodic fever syndromes

OBJECTIVES: Periodic fever syndromes (PFS) are rare and have variable clinical manifestations, leading to missed diagnoses. Familial Mediterranean fever (FMF), hyperimmunoglobulin D syndrome (HIDS) and tumor necrosis factor receptor-associated periodic syndrome (TRAPS) are among the most frequently occurring monogenetic PFS. In this retrospective case-control study, we developed a diagnostic prediction model using text from the electronic health record (EHR). METHODS: We identified potential PFS cases from the EHR of a tertiary care hospital and subsequently screened for Eurofever/PRINTO classification criteria for PFS. Controls consisted of patients with billing codes corresponding to overlapping diagnoses, and/or other codes for diagnoses with symptoms also occurring in PFS. We used text mining to capture 60 distinguishing features and trained a naïve Bayes model on the presence of these features to discriminate cases from controls. External validation was performed on a secondary care hospital cohort. RESULTS: 194 cases (145 FMF) and 14,435 controls were included in the development cohort; 120 cases (118 FMF) and 34,315 controls in the external validation cohort. The model showed an AUC ROC of 0.90 during internal validation, and 0.88 [95%CI 0.84-0.91] in the external validation cohort. Restricting external validation to pre-diagnostic data (50 PFS cases) substantially decreased performance (AUC ROC 0.66 [95%CI 0.56-0.75]. CONCLUSION: Text-mined EHR data contains valuable information to distinguish PFS from related diagnoses. Pending further validation in the pre-diagnostic setting, and a strategy to keep false positive rates manageable, machine learning models may contribute to identification of undiagnosed patients with PFS.