Calibrating Observational Health Record Data Against a Randomized Trial

Importance The conditions required for health record data sources to accurately assess treatment effectiveness remain unclear. Emulation of randomized clinical trials (RCTs) with health record data and subsequent calibration of the results can help elucidate this. Objective To pilot an emulation of the KEYNOTE-189 RCT using a commercially available electronic health record (EHR) data source. Design, Setting, and Participants This retrospective cohort study used an EHR database spanning from April 2007 to February 2023. Follow-up began on treatment initiation and proceeded until an outcome event, loss to follow-up, end of data, or end of study period (640 days). The population-based cohort was ascertained from EHRs provided by 52 health systems across the US. Eligibility criteria were defined as closely as possible to the benchmark RCT. Patients with non–small cell lung cancer initiating first-line treatment for metastatic disease were included. Patients with evidence of squamous non–small cell lung cancer, primary nonlung malignant neoplasms, or identified EGFR / ALK variations were excluded. Data were analyzed from June to October 2023. Exposures Initiation of first-line pembrolizumab and chemotherapy and chemotherapy alone. Chemotherapy in both groups was defined as a combination of pemetrexed and platinum-based (carboplatin or cisplatin) therapy. Main Outcomes and Measures Outcomes of interest were 12-month survival probability and mortality hazard ratio (HR). Results A total of 1854 patients (mean SD age, 63.7 9.6 years; 971 52.4% men) were eligible, including 589 patients who initiated pembrolizumab and chemotherapy and 1265 patients who initiated chemotherapy only. The cohort included 364 Black patients (19.6%) and 1445 White patients (77.9%). The 12-month survival probabilities were 0.60 (95% CI, 0.54-0.65) in the pembrolizumab group and 0.58 (95% CI, 0.55-0.62) in the chemotherapy-only group, compared with 0.69 (95% CI, 0.64-0.74) in the KEYNOTE-189 pembrolizumab group and 0.49 (95% CI, 0.42-0.56) in the KEYNOTE-189 chemotherapy-only group. The mortality HR was 0.95 (95% CI, 0.78-1.16), compared with 0.49 (95% CI, 0.38-0.64) in the KEYNOTE-189 RCT. Conclusions and Relevance In this cohort study piloting an RCT emulation, results were incongruous with the benchmark trial. Differences in patient treatment and data capture between the RCT and EHR populations, confounding by indication, treatment crossover, and accuracy of captured diagnoses may explain these findings. Future feasibility assessments will require data sources to have important oncology-specific measures curated.