Validation of the US Food and Drug Administration's COVID‐19 Disease Severity Categorization for Use in Real‐World Data

BACKGROUND: The US Food and Drug Administration proposed a five-level disease severity categorization for baseline COVID-19 (asymptomatic, mild, moderate, severe, critical). AIMS: We conducted a pilot study aimed to validate the performance of the ICD-10-CM diagnosis code for COVID-19 (U07.1) and operational definitions (code-based algorithms) for each disease severity category in Medicare administrative data. MATERIALS & METHODS: We sampled 250 community-dwelling beneficiaries aged ≥ 18 years with a U07.1 code on an inpatient hospital or ambulatory claim between April 1, 2020 and June 18, 2022. We used stratified sampling based on care setting and COVID-19 treatment status and adjusted results using sampling weights. Using medical records as the reference standard, we calculated positive predictive values (PPV) and 95% confidence intervals (CI). We conducted prespecified secondary analyses to improve algorithm performance by using refined disease definitions. RESULTS: We received medical records for 190 (77%) beneficiaries. Of these, 171 had a positive SARS-CoV-2 test result. The PPV of the COVID-19 diagnosis code was 89% (CI: 83%-93%) overall, 88% (CI: 80%-93%) in the ambulatory setting, and 93% (CI: 82%-97%) in the inpatient setting. The operational definition for disease severity varied in performance by severity level and measurement strategy. The best performance was: PPV for asymptomatic and mild combined 69% (CI: 59%-77%), PPV for moderate 58% (CI: 34%-78%); PPV for severe 42% (CI: 27%-59%); and PPV for critical 85% (CI: 57%-96%). DISCUSSION: The code, U07.1, reliably identified beneficiaries with COVID-19 in both ambulatory and inpatient settings. The operational definition to classify COVID-19 disease severity notably improved performance when we implemented selected refinements. CONCLUSION: Researchers should consider the moderate performance of the proposed operational definitions when using administrative claims data to assess COVID-19 disease severity.