Clarifying NAP5

We thank Professors Avidan and Sleigh for their editorial 1 accompanying the summary papers of the 5th National Audit Project (NAP5) 2-4. They rightly stress the novel results we have been able to present, along with new pathways of care to reduce the incidence and impact of accidental awareness during general anaesthesia (AAGA). However, there are some comments within their editorial that need clarification and, at times, correction. First, they make what we feel is a common error of discussing ‘incidence’ in relation to AAGA, as if AAGA were a single entity or its incidence a single quantity. This invariably leads to a discussion of whether the ‘Brice incidence’ (~1:600) is more ‘accurate’ than a ‘NAP5 incidence’ (~1:19 000) and they clearly express that they prefer the former. What they have overlooked is that AAGA is a highly heterogenous condition and as such, varies greatly in both its impact upon patients and its incidence. Thus to correct the data they quote, the NAP5 estimates of ‘incidence of reports’ actually lie between 1:670 (for obstetrics; remarkably close to a Brice incidence) to 1:136 000 (when no neuromuscular blockade is used). These wide estimates do not reflect some imprecision on our part, but are very robust data and indicate to us that AAGA is not a single entity 5, 6. The ‘doughnut’ diagram in Absalom and Green's editorial 7 (itself based upon a bar chart presented elsewhere 8) nicely encapsulates this notion. Different methods (Brice vs NAP5) detect different aspects of a heterogenous condition. The limitations of examining only spontaneous reports of AAGA are self-evident and we have stressed them at numerous points in the NAP5 project 9, 10. Similarly, there are numerous potential limitations to the Brice questionnaire in terms of both its use in formal studies and a lack of transparency over how the answers to the questionnaire are then interpreted. In our view, most will be gained by accepting both Brice and NAP5 estimates, and then trying to understand how the two methods relate to clinically relevant occurrences of AAGA, by further examining the uncertainty around each of them. Avidan and Sleigh suggest that the denominator data (from the NAP5 Activity Survey) is indirectly inferred and had an undisclosed percentage error. This is incorrect, as our multinational survey was specifically conducted to determine a denominator 11, 12 and our paper specified the formula for estimating the upper 95% CI for the fractional error (see Supplementary Figure S18 in reference 12 for the graphical presentation). For our Activity Survey sample size of > 20 000, this was at worst just 2% (upper 95% CI). Avidan and Sleigh also criticise the limitations of our numerator data, repeating limitations that we have ourselves described 2-4. We can clarify that reporting awareness was not dependent on publicity about NAP5 – only 1% of patients were aware of NAP5 when they reported – and it was both intentional and necessary that we included new reports of historical cases as to omit these would undeniably have led to an underestimate of the incidence of reports. It is worth noting that, despite any such concerns, our cohort probably represents a larger number of cases of AAGA than all previous reports of AAGA combined that have ever been reported in the literature. Furthermore, the NAP5 cases have been analysed with a depth and transparency often missing from many studies that use the Brice methodology. We are surprised by Avidan and Sleigh's apparent reluctance to embrace our suggested anaesthesia-specific checklist, to be incorporated into the WHO checklist. They feel this is an unacceptable imposition on an anaesthetist's time. We think they have overlooked our important and novel result that over half of all AAGA cases arose at or after induction, before commencement of surgery, while 40% of those arising ‘during surgery’ occurred at skin incision and had causal factors arising during induction. Almost all previous studies of AAGA appear to have focused only on that minority of cases arising during surgery. End-tidal alarms (or specific depth of anaesthesia monitors) may not help in preventing the majority of the cases we discovered at induction or elsewhere – almost 80% of NAP5's AAGA reports occurred when use of end-tidal alarms would have been ineffective or impractical – and a checklist seems the rational way forward. The WHO checklist is now firmly embedded in our practice, but lacks anaesthesia-specific steps. This small addition will not only prevent AAGA but also underline the anaesthetist's essential role as a key member of the wider surgical team in promoting safety. We recognise that Avidan and Sleigh wrote their editorial before the full NAP5 Report was released. It is always difficult to write editorials in such circumstances. It is inevitable that the published papers are only summaries – and we recommend those wishing for clear detail to read also the full Report (available at http://www.nationalauditprojects.org.uk/NAP5report). There is more than one acceptable methodology for studying AAGA and all are likely to have limitations. NAP5 is the most patient-focused study of AAGA ever performed: mapping the precipitants, risk factors, experiences and consequences of AAGA in a way that has never been possible before. We look forward to further discussion on the causes of the apparent chasm between the incidence of reports from NAP5 and that using the Brice questionnaire, and their clinical relevance.