What is the clinical evidence from this study?

Study design: Review. Population: Cardiovascular disease (n=50). Primary outcome: Proportion of studies ignoring the multicenter nature of data in statistical analysis.

What does this research mean for the field?

The vast majority of multicenter cardiovascular risk prediction studies ignore the clustered nature of their data in statistical analyses and lack transparency in reporting generalizability. Novelty: ClaimNovelty.SYNTHESIS. Consensus alignment: ConsensusAlignment.NEUTRAL.

February 22, 2019Open Access

Untapped potential of multicenter studies: a review of cardiovascular risk prediction models revealed inappropriate analyses and wide variation in reporting

Key Result

In a review of 50 multicenter cardiovascular risk prediction studies, 39 (78%) ignored the clustered nature of the data in their statistical analysis.

Structured PICO

Population

50 randomly selected multicenter studies published after 2000 from the Tufts PACE Clinical Prediction Model Registry containing published clinical prediction models for patients at risk for and with known cardiovascular disease.

Outcome

Properties, analysis, and reporting of multicenter studies, including statistical handling of clustered data and reporting of generalizability.

Regression methods and validation strategies tailored to multicenter studies are underutilized in cardiovascular prediction research, and reporting on generalizability lacks transparency.

Limitations

Limited to 50 risk prediction studies in cardiovascular disease.
Publication bias may have influenced the results, potentially leading to overestimated study sizes and numbers of participating centers.

Abstract

BACKGROUND: Clinical prediction models are often constructed using multicenter databases. Such a data structure poses additional challenges for statistical analysis (clustered data) but offers opportunities for model generalizability to a broad range of centers. The purpose of this study was to describe properties, analysis, and reporting of multicenter studies in the Tufts PACE Clinical Prediction Model Registry and to illustrate consequences of common design and analyses choices. METHODS: Fifty randomly selected studies that are included in the Tufts registry as multicenter and published after 2000 underwent full-text screening. Simulated examples illustrate some key concepts relevant to multicenter prediction research. RESULTS: Multicenter studies differed widely in the number of participating centers (range 2 to 5473). Thirty-nine of 50 studies ignored the multicenter nature of data in the statistical analysis. In the others, clustering was resolved by developing the model on only one center, using mixed effects or stratified regression, or by using center-level characteristics as predictors. Twenty-three of 50 studies did not describe the clinical settings or type of centers from which data was obtained. Four of 50 studies discussed neither generalizability nor external validity of the developed model. CONCLUSIONS: Regression methods and validation strategies tailored to multicenter studies are underutilized. Reporting on generalizability and potential external validity of the model lacks transparency. Hence, multicenter prediction research has untapped potential. REGISTRATION: This review was not registered.

Mark Helpful

Bookmark

Relay

View Full Paper