Concurrent, Construct, and Predictive Validity of a Claims-Based Algorithm for Prenatal Care Utilization Against Birth Certificate Records

A previously published algorithm validated its accuracy in identifying prenatal care as a predictor for adverse birth outcomes, using claims data systematically. This study aims to validate that algorithm for prenatal care initiation and frequency against birth certificates, assessing concurrent, construct, and predictive validity for agreement, bias, and risk identification. Data sources were South Carolina Medicaid claims and state birth certificates, linked by a de-identifiable maternal identifier. This retrospective cohort study used data spanning 2016 to 2021 (72,822 deliveries). We compared algorithm estimates with birth certificate records, examined disagreement sources, and assessed predictive validity using a logit-binomial generalized estimating equation model with robust standard errors. The algorithm estimated first PNC initiation 1.32 months earlier than birth certificates (p < .001), with poor absolute agreement (CCC = 0.31) and wide limits of agreement (− 4.71 to 2.07 months). Total visits showed minimal mean bias (− 0.12 visits) but individually deviated by ± 10 visits. Discrepancies varied by care timing, inpatient/emergency department use, and provider type. Predictive validity was similar across sources for cesarean sections and low birth weight, though algorithm estimates yielded slightly higher area under curve for preterm birth (difference = 0.0111, p < .001). Algorithm estimates cannot replace birth certificate records for measuring prenatal care utilization. The algorithm better captures encounter frequency, especially among underserved populations using Public Health Clinics, but provides limited service content information. Future studies could combine both: the algorithm for frequency and birth certificates for clinical content, offering a comprehensive measurement of timing, frequency, and content.