Validation of a risk assessment instrument for pulmonary arterial hypertension

This editorial refers to ‘A comprehensive risk stratification at early follow-up determines prognosis in pulmonary arterial hypertension’†, by D. Kylhammar et al., on page 4175. In this issue of the journal, Kylhammar et al.1 aimed to validate the pulmonary arterial hypertension (PAH) risk stratification instrument presented in the 2015 ESC/ERS guidelines.2 , 3 For 530 patients from the Swedish PAH registry, they calculated a risk score by assigning a grade from 1 to 3 for each available variable according to the risk level of that variable (low risk = 1, intermediate risk = 2, high risk = 3) and obtaining the mean grade for each patient, rounded to the nearest integer, thereby defining their risk group (low, intermediate, or high) at baseline. They then recalculated the mean grade at the time of follow-up for 383 patients who had a follow-up assessment within 1 year of diagnosis. At baseline, the vast majority of patients were intermediate risk (67%), with 23% classified as low risk and 10% as high risk. Surprisingly, there was little change in the proportions of patients who were low risk (29%), intermediate risk (60%) or high risk (11%) at follow-up, after PAH treatment was initiated. They found that the application of their risk score approach predicted the 1-, 3-, and 5-year risk of death at baseline and follow-up. This held true in important subgroups with idiopathic, familial, and connective tissue disease-associated PAH, and when older idiopathic PAH patients (>65 years old) were excluded. The 1-year mortality rates in the low-, intermediate-, and high-risk groups at baseline were 1, 17, and 26%, respectively, with a mortality risk in the intermediate group somewhat higher than the expected 5–10% in the guidelines.2 , 3 Repeating the risk assessment at follow-up, the 1-year mortality rates for low-, intermediate-, and high-risk patients were 1, 9, and 30%, which correspond well to the estimates proposed in the guidelines. They also confirmed previous findings by demonstrating the importance of improvements and maintenance of low-risk features during follow-up.4 , 5 Patients who worsened from low risk at baseline to intermediate or high risk at follow-up had much worse survival than those who were intermediate or high risk at baseline and improved to a low-risk profile. Perhaps most importantly, survival consistently improved as an increasing proportion of available variables at follow-up were in the ‘low-risk’ profile. Validation of guideline recommendations in independent, real-world cohorts is a crucial step in guideline dissemination and implementation, and, in this vein, Kylhammar et al. succeed in demonstrating that the risk assessment instrument proposed in the 2015 ESC/ERS guidelines has prognostic utility.1–3 Importantly, their study illustrates the prognostic significance of response to therapy, irrespective of baseline values. Their results also suggest that long-term prognosis is better with an increasing number of variables in the low-risk range during follow-up, akin to a dose–response effect between low-risk features and survival. Thus, an optimal treatment response can be considered the achievement and maintenance of all clinical, functional, and exercise variables in the low-risk zone. However, the results from this study are a sobering reminder that, despite the progress made in treating PAH, significant functional impairment persists and survival remains poor for a large proportion of PAH patients. The risk stratification instrument in the 2015 ESC/ERS guidelines is based on variables and cut-off points derived from several studies, but the risk assessment tool itself has a level of evidence C (consensus of opinion of the experts, and/or small studies, retrospective studies, and registries).2 , 3 Although this study was also a retrospective registry analysis, it should increase confidence that this risk assessment instrument is valid in a real-world cohort in the modern treatment era. However, the efficacy of a goal-oriented approach remains to be tested in a prospective or randomized study. While Kylhammar et al. have validated the risk assessment in an incident cohort within the first year, it remains to be shown whether it performs as well at predicting short-term prognosis in prevalent cohorts, who may achieve a low-risk profile later on in the disease course (after 1 year) and who may differ substantially from incident patients.6 , 7 There are also several important limitations to consider, several of which are addressed in their Discussion. There was substantial loss in patient numbers between the baseline analysis cohort (n = 530) and the follow-up analysis cohort (n = 383), of which 53% were unexplained, which could potentially have introduced a selection bias. Only a third of patients assessed in the follow-up analysis had invasive haemodynamic measurements available for the risk assessment, which may limit generalizability of their follow-up validation. Although they provided separate analyses in important subgroups of idiopathic/familial and connective tissue disease-associated PAH patients, the low numbers of congenital heart disease, porto-pulmonary, and drug-induced PAH limits the generalizability to these subtypes. Although the proportion of patients achieving a low-risk profile at follow-up was low (29%), inclusion of patients began in 2008 before the publication of important studies supporting initial combination therapy.8–11 However, initial combination therapy and parenteral prostacyclins still appear to have been underused in this cohort. Only 4% of high-risk patients received prostacyclin monotherapy, and 44% received either dual or triple combination therapy. Excluding 4% who received a calcium channel blocker, this leaves nearly half of the high-risk patients receiving initial monotherapy with an oral PAH-specific medication. Initial dual (49%) or triple combination therapy (6%) was higher when excluding older patients >65 years old. However, there remained a significant proportion of these younger high-risk patients who appear to be undertreated. Even fewer intermediate-risk patients (17%) received combination therapy. Together, these results may explain the surprisingly low proportion of patients who achieved a low-risk profile at follow-up. The relatively incomplete haemodynamic follow-up data in this study (only 33% had a follow-up cardiac index measurement) raises questions as to the relative importance of invasive compared with non-invasive indices of right ventricular function.1 A non-invasive risk assessment approach has not yet been validated and presents potential problems. For example, for a 30-year-old patient with idiopathic PAH who has an N-terminal pro brain natriuretic petide (NT-proBNP) level of 290 ng/L, 6 min walk distance (6MWD) of 450 m, and New York Heart Association (NYHA) II functional class, the average score is 1 and the patient is classified as low risk without haemodynamic measurements. However, in young idiopathic PAH patients without co-morbidities, haemodynamics can often be discordant. If the same patient had a right atrial pressure of 9 mmHg, cardiac index of 2.3 L/min/m2, and an SvO2 of 59%, which results in a rounded averaged score of 2, they would be classified as intermediate risk and treatment might be escalated using a goal-oriented approach. Despite these important insights from Kylhammar et al., several questions regarding the 2015 ESC/ERS risk assessment remain.1–3 Are the goals proposed in the risk assessment tool the best treatment goals for all patient groups? The ‘EFORT’ study (Evaluation of prognostic FactORs and Treatment goals in PAH) is an ongoing prospective study that aims to clarify the combination of variables that are associated with the best survival.12 The variables associated with the best outcome in this study could also be considered as potential treatment goals. What is the simplest and most practical way to implement a prognostication tool that involves nine clinical, exercise/functional, imaging, and haemodynamic variables, which are not always available for each patient? The decision to use an average score rounded to an integer in Kylhammar et al.1 presents obvious challenges when trying to classify an individual into a risk category that may influence the decision to escalate PAH therapy. Furthermore, it assumes that equal weight should be given to all variables in risk assessment, and does not determine whether the presence of certain high-risk parameters negate the prognostic relevance of other low-risk variables. A recent study from our group using the French PAH registry found that, similar to Kylhammar et al., a simple quantification of the number of low-risk features for four clinical and haemodynamic variables was sufficient to predict prognosis at baseline and follow-up.13 In reality, patients do not neatly fall into a green, yellow, or red category. Rather, individual patients present with various permutations of low-, intermediate-, and high-risk features. One is usually left with a risk assessment profile resembling different iterations of a Rubik’s cube (Figure 1) in pursuit of aligning modifiable PAH prognostic variables in the ‘green category’, which is ultimately achieved by few patients. Non-modifiable prognostic factors such as PAH aetiology and gender also influence the risk of mortality, but are constant and not included in the risk assessment instrument. Advancing age, accumulation of co-morbidities, and treatment burden also affect symptoms, functional class, and exercise capacity, leading to varying combinations of risk assessment variables in each risk category that wax, wane, and evolve over time. The risk assessment table for pulmonary arterial hypertension (PAH) patients proposed in the 2015 European Society of Cardiology/European Respiratory Society2 , 3 is represented by the schematic table at the top. This risk assessment instrument comprises several modifiable variables that are independently predictive of mortality in PAH and which broadly reflect clinical measures of symptoms, exercise capacity, or indices of right ventricular function. For each variable, values may be predictive of a low (<5%), intermediate (5–10%), and high risk (>10%) of 1-year mortality. Application of theory and clinical guidelines to individual patients is less straightforward, as PAH patients may have some variables in the green ‘low-risk’ category, other variables in the yellow ‘intermediate-risk’ category, and some variables with ‘high-risk’ values, leaving a clinical risk assessment resembling a Rubik’s cube (left cube). Non-modifiable prognostic factors such as PAH aetiology and gender affect the risk of mortality but are constant and do not change with treatment. Optimally, PAH treatment results in more of the modifiable variables improving to the green low-risk zone; however, few patients achieve this1 (middle cube). Over time, clinical worsening and disease progression often occur, as does the accumulation of co-morbidities, higher treatment burden, and ageing, which may contribute to an increased proportion of variables outside of the green low-risk zone (right cube). Thus, treatment of PAH should be goal directed and aggressive, but also needs to be personalized to the individual patient based on their individual risk assessment profile, with attention to the impact of other non-modifiable prognostic factors, age, co-morbidities, and total treatment burden (bottom, arrows). The authors also propose that their risk assessment composite score may provide a useful endpoint in future clinical trials,1 which does indeed have merits, since it is less complicated than other validated risk assessment tools such as the REVEAL score.14 , 15 However, these risk assessment tools have never been directly compared. Further understanding of the relative prognostic importance of each variable and evaluating the validity of a non-invasive version of the risk assessment instrument could further simplify the 2015 ESC/ERS risk assessment instrument. Furthermore, confirmation of the validity of the approach used by Kylhammar et al. in other cohorts, PAH subgroups, and in prospective studies will be necessary before widespread adoption of the 2015 ERS/ESC risk assessment as a clinical trial endpoint. Conflict of interest: J.W. has received a long-term research fellowship (LTRF 2015-4780) from the European Respiratory Society/Canadian Thoracic Society during the current work.