Risk adjustment for cardiac surgery in adults with congenital heart disease: what do we know and what do we need to learn?

The European Congenital Heart Surgeons Association (ECHSA) is to be congratulated for another meaningful contribution to the art and science of outcomes analysis and quality improvement in cardiothoracic surgery [1]. As of 2000, more adults than children are alive with congenital heart disease [2], and each year, more of these adults with congenital heart disease (ACHD) undergo surgery. Furthermore, many of these operations are reoperations in ACHD and are associated with unique risks [3]. Previous analysis of the Society of Thoracic Surgeons Congenital Heart Surgery Database (STS CHSD) reveals that a history of cardiac surgery does not independently confer a significant incremental risk of Operative Mortality, but that patients with greater numbers of previous operations appear to be at higher risk [3]. In this issue of The European Journal of Cardiothoracic Surgery, ECHSA has reported an analysis [1] of their ECHSA Congenital Database (ECHSA CD) that evaluates the predictive power of two scoring systems for ACHD undergoing congenital heart operations entered into the ECHSA CD: The Society of Thoracic Surgeons—European Association for Cardio-Thoracic Surgery Congenital Heart Surgery Mortality Categories (STAT Mortality Categories) [4, 5] The STS Adult Congenital Heart Surgery (ACHS Mortality Score) [6, 7, 8]. In this important analysis of ECHSA DB, a total of 17 638 operations (99.86% of all operations performed in ACHD in ECHSA CD) were assigned STAT Mortality Scores, and the Operative Mortality for STAT classified operations was 2.27%. A total of 17,214 operations (97.46% of all operations performed in ACHD in ECHSA CD) were assigned an ACHS score, and the Operative Mortality for ACHS classified operations was 2.07%. The discriminatory power of the STAT and ACHS scores was assessed using the area under the receiver operating characteristics curve (c-index). The c-index for Operative Mortality was 0.720 for the STAT Mortality Score and 0.701 for the ACHS Score. The authors concluded that the ‘ACHS Mortality Score and STAT Mortality Score reached similar, moderate predictive power in adult patients undergoing congenital heart surgery in ECHSA database’. The authors provided the following ‘take- home message’: ‘ACHS and STAT scores perform moderately. These procedure-based scores might give better results if patient-specific factors were included’. Because case mix can vary from program to program, meaningful programmatic analysis of outcomes necessitates characterization of case mix [9]. Both Risk Stratification and Risk Modelling are methods that are commonly used to account for variations in case mix; and therefore, both of these methodologies facilitate the analysis of pediatric and congenital cardiac surgical outcomes [9]. Risk stratification is a method of analysis in which the data are divided into relatively homogeneous groups (called strata) [4, 5]. The data are analysed and reported within each stratum. Three methods of risk stratification have been used in ECHSA CD and STS CHSD: Aristotle Basic Complexity Levels (ABC Levels). Risk Adjustment for Congenital Heart Surgery-1 Categories (RACHS-1 Categories). STAT Mortality Categories [4, 5]. Since 2010, the primary method of risk stratification used in ECHSA CD and STS CHSD is the STAT Mortality Categories [4, 5]. The STAT Mortality Categories are an empirically derived method of risk stratification based on the statistical estimation of the risk of mortality from an analysis of objective data from STS CHSD and ECHSA CD. The STAT Mortality Score and STAT Mortality Categories were developed based on an analysis of 77 294 operations entered in the STS CHSD and the ECHSA CD (ECHSA CD provided 33 360 operations and STS CHSD provided 43 934 operations). Procedure-specific mortality rate estimates were calculated using a Bayesian model that adjusted for small denominators. Operations were sorted by increasing risk and grouped into 5 categories that were designed to minimize within-category variation and maximize between-category variation. In 2015, Fuller et al. published the ACHS Mortality Score [6, 7]. In this publication [6], in-hospital mortality was analysed for the 152 most common procedures/procedural groups in adults 18 years of age and older in the STS CHSD (2000–2013). Procedure-specific adult mortality rate estimates were calculated using Bayesian methods and adjusting for small denominators for procedures with 30 cases or more (N = 52). Each procedural group was assigned an ACHS Mortality Score ranging from 0.1 to 3.0 based on the estimated mortality rate. Discrimination was assessed using the c-index in a separate validation sample. In this analysis by Fuller and colleagues, the c-index for the ACHS Mortality Score was 0.809 whereas the c-index for the ‘non-age-specific’ STAT Mortality Score applied to adults was 0.777. In 2018, Hörer and colleagues published an evaluation of the predictive power of the STS ACHS score for 1,654 consecutive operations performed in ACHD at two European centers and concluded that “The ACHS score reached similar, good predictive power in two different centers. The score is a useful tool to analyze surgical outcomes and to support individual decision making” [8]. The STAT Mortality Scores and Categories were originally developed in 2009 [4]. Recently, STS CHSD published an update of the STAT Mortality Categories with the publication of the STAT Mortality Score 2020 and the STAT Mortality Categories 2020 [10]. This update was designed to accomplish four objectives: (i) provide updated empirically derived STAT Mortality Scores and STAT Mortality Categories reflective of contemporary outcomes (for operations previously assessed in the STAT Mortality Score 2009 and the STAT Mortality Categories 2009) [4, 5], (ii) provide empirically derived STAT Mortality Scores and STAT Mortality Categories for operations added to the STS CHSD and ECHSA CD since 2009 (iii) utilize Operative Mortality as an endpoint instead of Discharge Mortality as an endpoint, and (iv) provide empirically derived STAT Mortality Scores and STAT Mortality Categories for operations with multiple component procedures for which the statistically estimated risk of mortality is different from that of the highest risk component procedure. Similarly to the STAT Mortality Score 2009 and the STAT Mortality Categories 2009, the STAT Mortality Score 2020 and the STAT Mortality Categories 2020 apply to neonates, infants, children and, adults with congenital heart disease. Similarly, the STS Morbidity Score and STS Morbidity Categories [11] apply to patients of all ages undergoing surgery for congenital heart disease and stratify procedures based on the statistically estimated risk of major postoperative complications and prolonged length of stay. Unlike risk stratification, a risk model is a mathematical formula that can be used to estimate the probability that a given outcome will occur [9], typically based on data from a large benchmark population. Risk models are developed for different purposes and different types of analyses. The intended use of risk models may be to estimate the risk of a given patient experiencing a particular outcome, to assess the case mix of a given program, or to assess the performance of a given program in relation to its own unique case mix, within the context of multi-institutional aggregate outcome data [9]. The point in time at which risk is being assessed with a risk model is critical in each of these applications [9]. The STS CHSD Mortality Risk Model [12–14] facilitates estimation of the risk of Operative Mortality, adjusted for preoperative patient-level factors present at the time that the patient enters the operating room. The primary use of the STS CHSD Mortality Risk Model [12–14] is to adjust for case mix when estimating hospital-specific risk-adjusted mortality rates of STS CHSD participants in relation to the aggregate STS benchmark. We agree with the ‘take-home message’ provided by ECHSA that the procedure-based scores analysed for ACHD in their recent analysis [1] ‘might give better results if patient-specific factors were included’. This outcome has certainly been the experience when estimating and adjusting for risk across all age categories combined. Currently, in the STS CHSD Mortality Risk Model, age is modelled as a piecewise linear function with separate intercepts and slopes for each STS-defined age group (neonate, infant, child, adult). In addition, the model adjusts for each combination of primary procedure and age group (of which there are > 600). Thus, the interaction between procedure type and age group is accounted for when the model is used to provide statistically estimated rates of expected mortality for a given centre. In the past, efforts have been made to harmonize the STS CHSD and the STS Adult Cardiac Surgery Database (ACSD) in order to assure that appropriate variables are present in the STS databases to achieve this objective of including appropriate patient specific factors when estimating risk in ACHD. Recently, two of us (JSN and FGS) have directed an initiative to identify opportunities for enhanced collection of data about ACHD within the STS databases. The rationale for this initiative was that (i) each of the current STS databases may lack some of the necessary granularity to capture patient-level variables relevant to the growing population of ACHD and (ii) these variables should be harmonized across the STS CHSD and the STS ACSD. Consequently, a structured review of existing variables in the STS CHSD and the STS ACHD was conducted. The review strategically followed the update of the STS ACSD that occurred in the summer of 2020, with the aim of synchronizing this review with the planned update of the STS CHSD in the winter and spring of 2021. This update will facilitate the eventual addition of patient-specific factors to risk models for cardiac surgery in ACHD. ECHSA is to be congratulated for enhancing our understanding of risk adjustment for ACHD undergoing cardiac surgery. This important contribution by ECHSA [1] is another step in the iterative process of using data to improve the care of adults with congenital heart disease.