Key result
Depth of anaesthesia monitors were estimated to reduce unintended awareness compared to standard clinical monitoring, supporting their clinical and cost-effectiveness in NICE guidance.
Why the study?
Do EEG-based 'depth of anaesthesia' monitors reduce the risk of unintended awareness and adverse outcomes compared to standard clinical monitoring in patients undergoing anaesthesia?
Do EEG-based 'depth of anaesthesia' monitors reduce the risk of unintended awareness and adverse outcomes compared to standard clinical monitoring in patients undergoing anaesthesia?
Answers are grounded in this paper and cite their sources.
This editorial discusses UK NICE guidance which suggests that depth of anaesthesia monitors may reduce unintended awareness, particularly during total intravenous anaesthesia (TIVA), though their cost-effectiveness is highly sensitive to baseline awareness rates and equipment costs.
The UK National Institute for Health and Care Excellence (NICE) published a guidance document on three commercially available ‘depth of anaesthesia’ monitors in November 2012 1. The guidance reflects an attempt to aim for certainty in an uncertain world. In this editorial, we will try to set the guidance into context by dealing with what is certain, but also with what is uncertain. The guidance process was initiated by a notification to NICE by GE Healthcare, the manufacturer of the E-Entropy device. In response, the NICE Medical Technologies Advisory Committee identified E-Entropy as potentially suitable for evaluation by the Diagnostics Assessment Programme. The initial scope of the evaluation was considered at a workshop in September 2011, to which relevant stakeholders were invited to contribute. These included specialist societies, patient groups and device manufacturers. However, these groups (including device manufacturers 2) had no direct input into the content of the guidance document. During the workshop, the evaluation was broadened to consider two other EEG-based ‘depth of anaesthesia’ monitors. It was also necessary to decide, in line with the programme manual for the NICE Diagnostic Assessment Programme 3, on a ‘comparator’. The comparator should reflect the technologies currently routinely used in the NHS, and is used as a benchmark with which the assessed technologies are compared. In this instance, the comparator technology was ‘standard anaesthetic monitoring’, defined as the combination of routine clinical observation and electronic monitoring used in everyday clinical practice to assess the adequacy of anaesthesia. A systematic literature review and economic analysis (the Diagnostic Assessment Report 4) was then commissioned to inform the process. Both clinical effectiveness and cost-effectiveness were considered. Clinical effectiveness was defined as the ability to reduce the risk of awareness and other adverse outcomes during anaesthesia. Cost-effectiveness explored health economic considerations such as the impact of these technologies on the sequelae of unintended awareness, side-effects of anaesthesia, time to awakening, time in the recovery room, and the impact on the use of anaesthetic agents, against the cost of the devices. The guidance therefore considered all adverse outcomes from anaesthesia, not just unintended awareness, which explains the perceived discrepancy between statements in the guidance document and the Diagnostic Assessment Report 2. The National Institute for Health and Care Excellence uses Incremental Cost-Effectiveness Ratios (ICERs) to guide health economic evaluations, supplemented by social value judgements and expert guidance. The ICER is the ratio of the incremental cost of an intervention to the benefit accrued in terms of quality-adjusted life years. As the ICER increases, the evidence base for any recommendations needs to be increasingly robust, especially at values above £30 000 (€34 800; $44 700) 3. There were three further meetings: one subgroup meeting to refine the scope and protocol for the literature review; and two meetings of the whole Diagnostics Advisory Committee to review the evidence and formulate the guidance document. The draft guidance was released for public consultation in August 2012 and, after reviewing comments from all stakeholders, including the Association of Anaesthetists of Great Britain and Ireland (AAGBI) and the Royal College of Anaesthetists (RCoA), the final guidance was published in November 2012. Some things, however, are less clear: ‘Depth’ is metaphorically apt 5 but quantitatively hollow, as ‘depth of anaesthesia’ monitors do not measure either ‘depth’ or consciousness, but rather dose-related changes in EEG parameters derived using proprietary algorithms that can be used to estimate the probability of unconsciousness. One of the challenging aspects of the assessment was deciding on a comparator technology that is in routine clinical use. The concept of ‘depth’ of anaesthesia has not been proven, and we are currently unable to measure consciousness, so there is no conventional comparator technology. Whilst the isolated forearm technique is considered to be the ‘gold standard’ for conscious responsiveness by many experts in the field of unintended awareness under anaesthesia 6, 7 it is not in routine clinical use 8, 9, so the comparator chosen for the assessment was therefore ‘standard clinical monitoring’. Most large prospective studies of unintended awareness report an incidence of about 0.15% (1–2 in 1000) 10, 11, with an ~80% reduction in high-risk cases when BIS monitoring is used 12, 13. However, an incidence of 1:15 000 accords better with anecdotal experience 8 and audited protocol-guided practice 14. The clinical effectiveness estimates in the Diagnostic Assessment Report were based on a modelled incidence of 0.16% for the general population and 0.45% for the higher-risk population, with a pooled odds ratio for reduction with depth of anaesthesia monitors of 0.45. These estimates are summary values from the meta-analysis 4, though the reported incidence varies widely depending on methodology and patient group 8, 10-15. This variability is reflected in the dependence of the cost-effectiveness analysis in section 5.34 of the guidance on the actual incidence of awareness used in the model. If the probability of awareness is set at 0.006% (as in 14) then the modeled ICER for BIS is £82 900 (€96 300; $123 500) compared with £8027 (€9300; $12 000) if the probability of awareness is 0.12% (as in 10-13). However, we do not know the incidence of awareness without subsequent recall, or the impact of this on patients who experience it 16. There are no uniform criteria for defining patients at high risk of unintended awareness 17 or at high risk from excessive administration rates of anaesthetic drugs, since the published evidence in this area has significant heterogeneity. The Committee took advice from the Specialist Committee members on this point. Some examples from published studies were included in a note attached to the recommendations to explain the context in relation to this guidance, but not all of these examples have a robust evidence base. Depth of anaesthesia monitors have not been evaluated in terms of sensitivity and specificity like any other diagnostic test, as there is no agreed reference test 18 for consciousness. Calibration of depth of anaesthesia monitors is performed either against a set of surrogate markers, such as loss of response to command or loss of eyelash reflex, or by correlating the output against that of another depth of anaesthesia monitor (usually BIS) [for example 19, 20]. Since the performance of any diagnostic test depends on the prevalence of the condition in question, testing depth of anaesthesia monitors in high-risk groups will make them seem more effective than they would be in the general population 21. However, unintended awareness is an uncommon event, which is why high-risk populations tend to be studied. Depth of anaesthesia monitoring is not a test in isolation. Outcomes from anaesthesia depend not only on what the monitor shows, but rather on a complex intervention comprising anaesthetists' response to the monitor and to other clinical signs, modulated by their judgement of the likelihood of awareness or overdose 22. This is perhaps the anaesthetist's key professional skill and is extremely difficult to mimic by automation 23. Published studies of depth of anaesthesia have rather glossed over this, but this is the point of education and training in the use of these devices that the guidance has stressed as being of importance. The majority of research in this field has been on BIS, the evidence for which has already been the subject of a Cochrane review 24, so this was updated in the Diagnostic Assessment Report to include the 11 trials published subsequently. Although there is greater uncertainty of clinical benefit for Entropy and Narcotrend than for BIS, the Committee concluded that these monitors are broadly equivalent to BIS. (Narcotrend was possibly disadvantaged in this evaluation, as only English-language publications were considered in the meta-analysis). Furthermore, the overall quality of the evidence base for depth of anaesthesia monitors is poor, because of limitations of sample size and variation in methodology. Careful adherence to protocol-guided anaesthesia with a volatile agent, or using end-tidal agent concentration alarms set to 0.7 MAC, may reduce the incidence of awareness to a comparable level to that obtained using BIS monitoring 14, 25, 26. However, even this dose of agent may result in a ‘response to command’ with the isolated forearm, with no apparent explicit or implicit recall 27. Conversely, the same dose may increase morbidity in ‘at-risk’ patients, for example an increased incidence of cognitive dysfunction in the elderly 28. Minimal alveolar concentration is the EC50 for spinal reflex responsiveness rather than conscious level 29, 30, with MACawake calibrated to response to command in volunteers 30, so maintaining a particular MAC (or EC50/EC95) value will not completely prevent unintended awareness in patients having surgery. There is no comparable measure for end-tidal agent monitoring for TIVA, and there are numerous reports of unintended awareness due to failure of TIVA systems, so the Committee recommended that a depth of anaesthesia monitor should be considered when TIVA is used. Although all available ‘depth of anaesthesia’ technologies were considered during the initial scoping exercise, there was not a large enough primary evidence base for most of them, in respect of the target measures, to permit their inclusion. The evaluation required that the monitors had to be compared with standard clinical monitoring, which excluded comparative research that indicates that the majority of EEG-based monitors respond in a similar manner to changes in the administration rates of currently available anaesthetic agents. This does not mean that the Committee believe that these other devices do not work. Implementation of diagnostic guidance is not mandatory. However, the guidance reflects evidence-based practice, so it is expected that the guidance will be taken into account when making clinical decisions. Four factors are likely to influence uptake: The cost-effectiveness model used in the Diagnostic Assessment Report demonstrates that the ICERs for this evaluation are heavily dependent on the incidence of unintended awareness and the cost of the sensors for the depth of anaesthesia device. The costs of using BIS and Entropy are higher than that of Narcotrend, primarily because Narcotrend uses regular adhesive electrodes rather than bespoke sensors. This may reflect the different marketing strategies of the individual companies, but it may also reflect the cost of developing and exploring the use of this technology in multicentre trials that have primarily been carried out using BIS. New adopters of depth of anaesthesia technology may wish to explore the cheaper options, which we hope will result in the publication of good quality, stand-alone evidence for these technologies, and incentivise a reduction in costs for bespoke sensors in the future. However, the guidance should now give anaesthetic departments, or individual clinicians, an easier pathway into adopting depth monitoring if they wish to do so for particular groups of patients, complementing individual business plans and persuasive argument. Unfortunately, most anaesthetists have not previously viewed unintended awareness as a serious problem 31, 32, although a fifth to a half have experienced a case in their practice 8, 31, 32. Whilst most anaesthetists accept that clinical signs are unreliable, few believe that depth of anaesthesia monitors should be used for routine cases, a view supported by the guidance. However, given that we cannot be sure that a patient is unconscious, a ‘depth of anaesthesia’ monitor could be extremely useful for reducing adverse outcomes from anaesthesia in some circumstances, which the guidance is trying to encourage. Some form of depth of anaesthesia monitor is available in 62% of UK hospitals, but they are rarely used 8. This sceptical approach may therefore need to change before there is widespread adoption of depth of anaesthesia monitoring. The output of any depth of anaesthesia monitor should be meaningful to the user, and perhaps the uncertainty in this area contributes to the low uptake of depth of anaesthesia monitors. Although the devices are calibrated to clinical endpoints like loss of eyelash reflex or response to command, we do not know what numbers below these thresholds mean in terms of cerebral processing or effects on other organs. Patients expect to be completely unconscious during general anaesthesia, and may become more aware of the existence of this technology and request it for themselves. Would an individual anaesthetist be liable for an episode of awareness if he/she did not use one of these devices? At present there is a majority of anaesthetists that do not use them (satisfying the Bolam principle), the evidence base is relatively weak (passing the Bolitho test), and this guidance is a technology evaluation, not a clinical guideline 33. However, the guidance states that the devices that were evaluated are both clinically effective and cost-effective on the evidence considered. Although the guidance does not mandate the use of these devices, it is for individual anaesthetists or departments to decide whether they wish to use them, based on clinical judgement for individual patients in the context of available evidence. The Diagnostic Advisory Committee decided, following considerable debate, not to make any specific recommendations about further research. Undertaking research in this area of anaesthesia is complex, and the Committee considered that the current uncertainty in the evidence base did not justify a potentially long delay in the uptake of what is likely to be a beneficial technology to the NHS and, particularly, to patients. The Committee considered that manufacturers and clinical researchers should now undertake studies to demonstrate the effectiveness of Entropy and Narcotrend in reducing unintended awareness, and further evaluate all three monitors in reducing all adverse outcomes of general anaesthesia. The joint RCoA/AAGBI National Audit Project on Unintended Awareness (NAP5) is due to report shortly, and may provide a ‘real-life’ estimate of the incidence of awareness with recall. This estimate of awareness relies on new reports from anaesthetists 8, and from patients' volunteering the information to healthcare professionals, so is likely to be lower than that found in studies specifically designed to look for awareness with structured Brice interviews. The true incidence of awareness may still be subject to debate 34, as will uncertainty about what to tell patients 35. A multicentre Australasian study, the Balanced Anaesthesia Study 36, aims to determine the effect of ‘light’ versus ‘deep’ general anaesthesia on all-cause mortality at one year postoperatively in 6500 moderate- to high-risk patients having major non-cardiac surgery. This study will not report for at least five years, but if it confirms that ‘depth of anaesthesia’ has an influence on mortality 37, this might constitute a driver for the more widespread adoption of depth of anaesthesia technologies in view of the ‘triple low’ hypothesis (hypotension and low BIS values at unusually low end-tidal agent concentrations) 38. A standardised process makes for an objective approach, and allows the necessary assumptions and judgements to be made explicit, and the inevitable biases and preconceptions declared and accounted for. The Diagnostics Assessment Programme is relatively new, and is designed to perform rapid, complex evaluations of medical diagnostic technologies where the quality of evidence for patient outcomes is low and implementation may increase NHS costs 2. The short timescale for the Diagnostic Assessment process and the limited time for discussion are very different to what we are used to in the NICE Clinical Guideline methodology. Although the process is well defined, there is little time for face-to-face discussion of complicated issues. By elaborating on some of the factors that shaped the guidance, we hope that we have allowed anaesthetists to understand why the guidance says what it does. All the authors were specialist committee members for this NICE guidance and have consulted the Project Team Lead in the preparation of this editorial, but any opinions expressed are their own. JA has received a grant from the Neuroanaesthesia Society to carry out research into BIS monitoring. AS is sceptical about the value of ‘depth of anaesthesia’ monitoring. He has been taken out to dinner by Masimo, manufacturers of the ‘Sedline’ brain function monitor. DS has received honoraria and unrestricted educational grants from Vickers Medical, Datex-Ohmeda, and Medical Device Management. He was Moderator for the NAP5 project, advising Local Coordinators on borderline cases of unintended awareness; his views do not necessarily represent the views of the NAP5 project team.
No takes yet. Share an insight, caveat, or question.
Smith et al. (2013) conducted an editorial in General anaesthesia. Depth of anaesthesia monitors (BIS, E-Entropy, Narcotrend) vs. Standard clinical monitoring was evaluated on Unintended awareness (OR 0.45). Depth of anaesthesia monitors were estimated to reduce unintended awareness compared to standard clinical monitoring, supporting their clinical and cost-effectiveness in NICE guidance.
Synapse has enriched 10 related papers. Consider them for comparative context: