Objectives: To validate claims-based algorithms for identifying thyroid eye disease (TED) cases and assess whether machine learning can improve case identification in a large, linked real-world dataset. Methods: Using a large, linked database from Komodo Health® and Academy IRIS® Registry, we evaluated six rule-based algorithms incorporating Graves’ disease (GD), eye symptoms and signs. The IRIS Registry’s curated data, based on confirmed TED diagnoses from medical notes, served as the reference standard. Additionally, we developed supervised machine learning models using demographic, diagnostic, procedural, and medication data. Feature selection was performed using recursive feature elimination to rank predictive codes and construct a simplified, interpretable model. Cross-validation was used to assess model performance and compare performance with the rule-based algorithms. Results: The rule-based algorithms demonstrated a trade-off between sensitivity and specificity, with some achieving high specificity but limited sensitivity. Algorithm 1 had the highest sensitivity (48.7%) but lower specificity (59.9%) and PPV (75.8%). Algorithms 2–5 demonstrated higher specificity (87.2–93.5%) but lower sensitivity (17.8–27.0%). Algorithm 6 improved sensitivity (33.4%) compared to Algorithms 2–5 while maintaining high specificity (86.8%) and a strong PPV (86.7%). Machine learning models demonstrated similar trade-offs. One model achieved improved specificity (77.2%) with sensitivity of 49.3%, outperforming Algorithm 1 in specificity while matching its sensitivity. Another model maximized specificity (91.7%) and PPV (89.8%) at a reduced sensitivity of 28.5%. These results highlight the flexibility of machine learning models in adjusting performance to address different research objectives. Conclusions: This study evaluated existing rule-based algorithms for identifying TED cases in claims data, revealing trade-offs between sensitivity and specificity. Machine learning models provide additional flexibility, allowing performance to be tailored to specific research use cases. While no single method consistently outperformed others across all metrics, both rule-based and machine learning approaches demonstrated value in improving TED case identification using real-world data sources.
Ma et al. (Fri,) studied this question.