A systematic review of 49 clinical prediction models for valvular heart disease revealed substantial heterogeneity in external validation performance, with a median 27.1% decline in discrimination.
Systematic Review (n=49)
How do clinical prediction models for valvular heart disease perform in external validations?
Many clinical prediction models for valvular heart disease lack external validation, and those that are validated often show substantial declines in discriminatory performance, highlighting the need for rigorous external validation before clinical use.
Effect estimate: median percentage change -27.1%
Background While many clinical prediction models ( CPM s) exist to guide valvular heart disease treatment decisions, the relative performance of these CPM s is largely unknown. We systematically describe the CPM s available for patients with valvular heart disease with specific attention to performance in external validations. Methods and Results A systematic review identified 49 CPM s for patients with valvular heart disease treated with surgery (n=34), percutaneous interventions (n=12), or no intervention (n=3). There were 204 external validations of these CPM s. Only 35 (71%) CPM s have been externally validated. Sixty‐five percent (n=133) of the external validations were performed on distantly related populations. There was substantial heterogeneity in model performance and a median percentage change in discrimination of −27.1% ( interquartile range , −49.4%–−5.7%). Nearly two‐thirds of validations (n=129) demonstrate at least a 10% relative decline in discrimination. Discriminatory performance of Euro SCORE II and Society of Thoracic Surgeons (2009) models (accounting for 73% of external validations) varied widely: Euro SCORE II validation c‐statistic range 0.50 to 0.95; Society of Thoracic Surgeons (2009) Models validation c‐statistic range 0.50 to 0.86. These models performed well when tested on related populations (median related validation c‐statistics: Euro SCORE II , 0.82 0.76, 0.85; Society of Thoracic Surgeons 2009, 0.72 0.67, 0.79). There remain few (n=9) external validations of transcatheter aortic valve replacement CPM s. Conclusions Many CPM s for patients with valvular heart disease have never been externally validated and isolated external validations appear insufficient to assess the trustworthiness of predictions. For surgical valve interventions, there are existing predictive models that perform reasonably well on related populations. For transcatheter aortic valve replacement (CPM s additional external validations are needed to broadly understand the trustworthiness of predictions.
Wessler et al. (Fri,) conducted a systematic review in Valvular heart disease (n=49). Clinical prediction models (CPMs) was evaluated on Performance in external validations (discrimination) (median percentage change -27.1%). A systematic review of 49 clinical prediction models for valvular heart disease revealed substantial heterogeneity in external validation performance, with a median 27.1% decline in discrimination.