Develop and validate a fair machine learning model to identify patients with high data-continuity in electronic health records data

Objectives: Electronic health record (EHR) data discontinuity, defined as receiving care outside of a particular EHR system, may cause misclassification of study variables. We aimed to: (1) quantify misclassification across levels of EHR data discontinuity and identify an optimal continuity threshold, (2) develop a machine learning (ML) model to predict EHR continuity and optimize fairness across racial and ethnic groups, and (3) externally validate the EHR continuity prediction model using an independent dataset. Materials and Methods: We used linked OneFlorida+ EHR-Medicaid claims data for model development and Research Action for Health Network (REACHnet) EHR-Louisiana Blue Cross Blue Shield (LABlue) claims data for external validation. A Harmonized Encounter Proportion Score (HEPS), adapted from prior continuity metrics, was applied to quantify patient-level EHR data continuity and the impact on misclassification of 42 clinical variables. Machine learning models were trained using routinely available demographic, clinical, and healthcare utilization features derived from structured EHR data. Results: Higher EHR data continuity was associated with lower rates of misclassification. A HEPS threshold of approximately 30% effectively distinguished patients with sufficient data continuity. Machine learning models demonstrated strong performance in predicting high continuity (area under the receiver operating characteristic curve [AUROC] = 0.77). Fairness assessments showed bias against Hispanic group, which was substantially improved following bias mitigation procedures. Model performance remained robust and fair in the external validation. Discussion: Our study offers a practical metric for quantifying data continuity in EHR networks. The current ML model incorporating EHR-routinely collected information can accurately identify patients with high care continuity. Conclusions: We developed a generalizable data-continuity classification tool that can be easily applied across EHR systems, strengthening the rigor of EHR-based research.