Are NICE processes fit for the evaluation of new interventional procedures?

The controversy over the delayed issue of guidance by the National Institute for Health and Care Excellence (NICE) regarding the use of endovascular aneurysm repair (EVAR) for abdominal aortic aneurysm (AAA) illustrates many of the difficulties inherent in providing evidence-based advice to the National Health Service (NHS) in the UK. The procedures and devices used for EVAR have developed rapidly over the past 20 years, and many now consider it the first-line elective treatment for AAA in most circumstances. The procedure has been evaluated by NICE on several occasions through different processes. Following publication of the early results of the EVAR trials1, it was considered sufficiently safe and efficacious for use in the NHS (Interventional Procedures Guidance IPG163). The 2009 appraisal (Technology Appraisal TA167) recommended it as a treatment option for patients with unruptured infrarenal AAA, with some anatomical, clinical and demographic factors to be taken into account when making the decision. The guidance appeared partly to contradict the findings of the assessment report, which demonstrated that EVAR was unlikely to be cost-effective in fit patients in comparison with open surgical repair2. EVAR has been adopted widely in the NHS, now accounting for about 70 per cent of elective procedures, and is used increasingly for ruptured AAA. The concurrent introduction of aneurysm screening has increased the proportion of younger, fitter men undergoing elective aneurysm repair, the group least likely to be cost-effective, according to the economic modelling2. The draft NICE guidance issued in May 2018, suggesting that EVAR should not be offered to people with an unruptured infrarenal AAA, has proved controversial. The Interventional Procedures Advisory Committee considers only safety and efficacy, whereas technology appraisal and guidelines include detailed assessment of cost-effectiveness. However, NICE states that it does not subscribe to a specifically utilitarian approach, but focuses on ‘procedural justice’, which has amongst its principles scientific rigour, inclusiveness, transparency and independence3. NICE guidance describes, in considerable detail, the methods of technology appraisal, including the cost-effectiveness analysis4, specifying the perspective of the health and social care system, and a version of cost-effectiveness based upon the cost per quality-adjusted life-year (QALY). QALYs are based upon their preferred health outcome measure, the EQ-5D™ (EuroQol Group, Rotterdam, the Netherlands), a generic measure with five dimensions, each valued on a three-point scale and converted to a utility using a standard tariff. A ‘discounted cost-per-QALY’ is compared to a nominal ‘willingness-to-pay’ threshold of £20 000 (approximately €23 100, exchange rate 27 October 2019) per QALY, which, in certain circumstances, can be extended to £30 000 (approximately €34 700). Apart from concerns about aspects of health outcomes that are not captured adequately by EQ-5D™, other aspects of healthcare provision that are valued by society are excluded from the calculations. These include issues such as equity, burden of disease, wider societal impacts, autonomy, dignity, continuity of care, location of services and aspects of the process of care, such as the invasiveness of treatment. NICE considered including some of these issues in relation to ‘value-based pricing’ in 2014, but dropped the proposals following consultation5. Although there is no objective way to choose between methods of evaluation, the choice may have a significant impact on the technologies that appear advantaged or disadvantaged. Specific considerations are of particular relevance to interventional procedures. The process utility is the additional value that might be attached to particular modes of treatment, such as different routes of administration, outpatient versus inpatient treatment, or more or less invasive methods. Evidence6 suggests that these may be valued over and above any short-lived differences in health captured by instruments such as the EQ-5D™. NICE may consider such preferences through its deliberative processes, for example using a raised threshold to allow for the convenience of oral rather than parenteral administration. However, major surgical procedures, compared with minimally invasive or medical options, are of a different order of magnitude, and adjusting the threshold to allow for this may be unsatisfactory. Major surgical procedures may result in very different risk profiles compared with less invasive or conservative treatments. For someone in their eighties needing treatment for AAA, the risk of procedure-related mortality for open repair may be 5–10 per cent, with EVAR carrying about one-third of this risk7. This early advantage may be given far more weight than risks of complications and retreatment in the future. The strength of this preference is unlikely to be represented adequately by the discount rate of 3·5 per cent used for NICE economic evaluations8. Drugs are stable technologies with pivotal clinical studies before licensing, defining the formulation and dosage regimens. In contrast, interventional technologies usually evolve, creating difficulties in assessing and applying evidence of effectiveness to procedures that are in a state of flux, or devices that may no longer be current. A changing selection of EVAR devices and techniques may suit patients with specific clinical and anatomical characteristics. Thus, unlike drugs for which indications are clearly defined, clinicians are not simply learning a new procedure, but developing experience in how particular methods or devices will suit individual circumstances. Drug prices are fixed through various national mechanisms, and there is an opportunity for (indirect) negotiation through patient access schemes. There are no such fixed prices or opportunity for national negotiation for devices, and a significant aspect of the cost of procedures may relate to hospital and staffing resources, which vary between centres, may be affected by individual patient characteristics, and are subject to differences in practice. Choices between different modalities of treatment often have implications for wider aspects of service configuration, such as the location of services, training requirements, the need for capital equipment, shifts in workload or joint working between specialties. Such changes may result in significant sunk cost, or organizational changes associated with the new technology. Modelling of cost-effectiveness using the methodology and limits usually set by NICE has been consistent in finding that, for relatively young and fit patients, EVAR is unlikely to be cost-effective, and may be dominated by open repair. Most models have considered alternative scenarios and subgroup analyses, and suggest that there are older, less fit people, for whom EVAR is a cost-effective option2,9. Modelling for the draft guideline did not consider such alternative scenarios in detail. Identifying an appropriate subgroup is not easy, owing to the absence of accepted methods for risk scoring, lack of relevant anatomical and clinical data, and reluctance to use factors such as age and sex to determine treatment policies. The appraisal in 2009, despite evidence that the procedure was unlikely to be cost-effective on average2, allowed clinical discretion, but drew attention to factors that may be relevant to the decision. This appeared, in effect, to place responsibility for cost-effective decision-making on the individual clinician. This proved unsatisfactory for several reasons. Although those with clinical expertise are clearly important stakeholders in determining the guidelines under which new technologies are made available, it is important to separate their policy role from their dealings with individual patients. Patients expect that individual clinical advice is based upon the most effective treatment available, and to base such advice on budgetary considerations is likely to undermine the relationship of trust between clinician and patient. Such discretion is also likely to favour more informed and empowered groups within society10. In the case of EVAR it is clear that asking clinicians to make decisions based on cost-effectiveness was ineffective. There is no right or wrong method for making decisions about the distribution of scarce healthcare resources. With many competing demands, trade-offs are required between utilitarian approaches that might maximize differing measures of benefit, and other libertarian or egalitarian considerations. In a publicly funded healthcare system in a democratic society, it is reasonable to expect that the process results in decisions that broadly reflect societal preferences. There are a number of measures relating to NICE methods, or areas where further research is required, that might help achieve this aim. The primary outcome measure recommended by NICE for generating the utilities used in calculating QALYs is the tariff derived from the EQ-5D™. This lacks sensitivity to small changes in health status and does not address significant aspects of health that may be considered important. This has resulted in the development of a five-level version of the measure and the suggestion of various bolt-ons to address areas such as hearing and visual impairment11. NICE has not currently adopted these modifications. In addition to the health outcomes measured by EQ-5D™, processes of care may be very important to patients, particularly in relation to interventional procedures where options may include major invasive surgery. It is possible to measure the value that is put on aspects of process, and formally consider trade-offs against other aspects of outcome12. The use of a cost-effectiveness threshold, which is varied in certain circumstances, has resulted in a system that tends to favour certain technologies, particularly end-of-life drugs and highly specialized technologies. The deliberative process used for NICE decision-making, accounts for other factors by raising (never lowering) the threshold in certain circumstances. Additional factors are taken into account only in relation to the new technology under consideration and not in relation to any existing technology that is likely to be displaced. Disinvestment decisions do not undergo the same level of scrutiny as new technologies, and are often invisible to decision-makers and service users who may be affected. To include consideration of more sensitive or inclusive outcome measures, process utilities or wider impact requires that such criteria are extended to potentially displaced activities. Adjusting the threshold to account for such factors, assigns them greater value when considering high-cost, high-impact interventions. An alternative approach would be to calculate net monetary or health benefit (or cost). A committee might consider it worth a few additional pounds per patient for a less invasive procedure, but not several thousand pounds – a difference that may not be apparent when considering this in terms of incremental cost-effectiveness ratios and thresholds. NICE guidance is based upon evidence that is subject to considerable uncertainty, but clear guidance may inhibit further research. Evidence regarding EVAR has changed little since the original randomized trials, and it is unlikely that further randomized studies would be feasible. Data collected through the National Vascular Registry lack long-term follow-up, whereas routine data can reveal readmissions, retreatment and mortality, but do not include detailed clinical or anatomical information. If the modelled costs are accurate, it is likely that EVAR has cost the NHS in excess of an additional £100 million (approximately €116 million) since the appraisal in 2009. A decision at that time for coverage with evidence could, at a small fraction of this cost, have provided the evidence for detailed predictive models, which could be used to identify appropriate populations for the cost-effective use of EVAR. Unfortunately, the financial arrangements that separate clinical and research budgets may mitigate against decisions that are subject to further research and evidence collection. The author declares no conflict of interest. At the time BJS went to press, NICE has still not issued the final guidance on EVAR for aortic aneurysms, suggesting its draft advice from 19 months ago is being revisited. This issue of BJS explores the way NICE decisions are made (see also Carroll et al., p1769), and shows how determining the preferences of patients and the public about their potential treatment can inform these difficult decisions (see Wickramasekera et al., p1775). Once NICE has issued its final decision, we will comment further on the future role of EVAR for the treatment of aortic aneurysms. R. J. Hinchliff and J. J. Earnshaw Editors, BJS