The effects of misclassification in routine healthcare databases on the accuracy of prognostic prediction models: a case study of the CHA2DS2-VASc score in atrial fibrillation
Key result
Using routine healthcare data for the CHA2DS2-VASc score resulted in substantial predictor misclassification but did not affect overall model discrimination for mortality (c-statistic 0.685 vs 0.682).
Why the study?
Does misclassification in routine healthcare databases affect the predictive performance of the CHA2DS2-VASc score for mortality in patients with atrial fibrillation?
Population
2363 patients with atrial fibrillation in a prospective cohort in general practice in the Netherlands
Comparison
Computerized retrieved data from electronic… vs Manually collected data after scrutinizing all…
Design
Cohort
Follow-up
median 2.7 (IQR 2.3-3.0) years
Authors
Discussion
Loading...
Member takes
Implication
Routine data may suffice for CHA2DS2-VASc mortality models in AF; leaves open validation for other outcomes and populations.
Study Design
Type
Cohort (n=2,363)
Multicenter
Yes
Structured PICO
Does misclassification in routine healthcare databases affect the predictive performance of the CHA2DS2-VASc score for mortality in patients with atrial fibrillation?
P
Population
2363 patients with atrial fibrillation in a prospective cohort in general practice in the Netherlands
I
Intervention
Computerized retrieved data from electronic medical records (index predictors) for CHA2DS2-VASc score
C
Comparator
Manually collected data after scrutinizing all complete medical files (reference predictors) for CHA2DS2-VASc score
O
Outcome
Predictive performance for mortality, quantified by c-statistic and calibration
Main Result
Absolute Event Rate: 0.685% vs 0.682%
Substantial predictor misclassification in routine healthcare data has only a limited effect on the overall predictive performance of the CHA2DS2-VASc score for mortality in AF patients.
Limitations
- Information in free text fields leaves room for different interpretation
- Did not subject patients to any new clinical assessment, meaning some misclassification might have occurred in the reference data
- Only evaluated a single prediction model
- Used all-cause mortality as the outcome instead of stroke risk, which the CHA2DS2-VASc rule was specifically designed to predict
Cite This Study
Doorn et al. (2017) conducted a cohort in Atrial fibrillation (n=2,363). Routine healthcare data (ICPC codes) for CHA2DS2-VASc score vs. Manually verified data for CHA2DS2-VASc score was evaluated on Model discrimination (c-statistic) for all-cause mortality. Using routine healthcare data for the CHA2DS2-VASc score resulted in substantial predictor misclassification but did not affect overall model discrimination for mortality (c-statistic 0.685 vs 0.682).