Assessing EHR Cohort Fitness With Partially Paired Registry Data: A New Statistical Approach Applied to Maternal Health in OneFlorida +

Cohort discovery using electronic health records (EHR) supports research recruitment and pragmatic clinical trials; however, the validity of EHR-derived cohorts depends on whether patient characteristics recorded in EHR are representative of the target population. While methods exist for assessing internal validity, no published methods address continuous outcomes when EHR and reference population data partially overlap at the individual level. We present a novel statistical approach for assessing EHR data cohort fitness when partially overlapping data are available from an EHR and a registry. For continuous outcomes, we derived an explicit test accommodating unequal variances and correlation between paired records. For binary outcomes, we recommend generalized linear mixed models with a binomial distribution and logit link. We demonstrate the new approach using maternal health data from the OneFlorida+ Data Trust (n = 37,200) and Florida birth certificate records (n = 210,910) from 2021. OneFlorida+ EHR had higher rates of Black race identification, residence in areas of higher social deprivation, and pre-existing diabetes or hypertension, while showing lower probabilities of gestational complications. Maternal age, height, and cesarean delivery rates were not meaningfully different between data sources. Observed differences may reflect the academic health center composition of OneFlorida+ partner sites; however, substantially higher missingness in OneFlorida+ EHR than birth certificate records for race, ethnicity, and location of residence suggests that data capture differences play a role. This replicable statistical method provides researchers with quantitative tools to assess cohort fitness and is applicable to other clinical research networks where EHR and registry data overlap.