Anesthesia Safety: Model or Myth?

Received from the Department of Anesthesiology, Montefiore Medical Center and the Albert Einstein College of Medicine, Bronx, New York.IN 1999 the Institute of Medicine (IOM) published a report entitled To Err is Human: Building a Safer Health Care System. 1In that report, the Committee on Quality of Health Care in America for the IOM asserted, “Anesthesia is an area in which very impressive improvements in safety have been made.” In support of this assertion the Committee stated that anesthesia mortality rates have decreased from 2 deaths per 10,000 anesthetics administered in the 1980s to about 1 death per 200,000 to 300,000 anesthetics administered today. The reference for such “impressive” gains, however, does not identify the studies that led to this conclusion. 2Multiple sources, including the Committee on Healthcare in America, have attributed this dramatic decrease in anesthesia mortality to a variety of mechanisms including improved monitoring techniques, the development and widespread adoption of practice guidelines, and other systematic approaches to reducing errors. †1,3–6In so doing, anesthesiology has been established as a model of safety, and other specialties are encouraged to engage in similar risk reduction strategies.Because the implications of establishing anesthesiology as a model of safety can have a far-reaching impact on the allocation of scarce healthcare resources, it is imperative that the basis for these claims be critically examined. Consequently, this author reviews the medical literature pertaining to anesthesia-related mortality rates, published over the last 35 years, with a focus on methodology and operational definitions applied by the various investigators. More recent perioperative mortality data, collected from two university-based anesthesia practices from January 1, 1992 through December 31, 1999, are also introduced.The Medline and HealthStar databases were searched using subject keywords “anesthesia AND mortality” from 1966 to 2000 and from 1975 to 2000, respectively. Publications were included in this review if their titles or abstracts were available in English and suggested a perioperative mortality rate related to anesthetic management in a general patient population over a specified period of time based on original data. Publications were excluded if the anesthetic management was limited to a particular technique, or the patient population was limited to a particular procedure, associated disease state, or age group. Each publication was then reviewed as needed to identify author(s), study period, data source, perioperative mortality rate, anesthesia-related mortality rate, mortality rate for which anesthesia was solely responsible, and preventable anesthetic mortality rate as defined by each study.Anesthesia-related mortality rates taken from the literature review were plotted against each study's midpoint on an “attribute P chart.” Attribute P charts reflect the number of defectives (anesthesia-related deaths) as a proportion of variable sample size. 7Studies with the same midpoint of the time period under investigation were combined into one sample, so that one data point could be generated for a single point in time. Upper control limits (three standard deviations from the average proportion defective) were established based on a binomial distribution. Systems were considered “out of control” if a point fell outside of the control limits or a “run” or “trend” was detected. A run is defined as a succession of seven points that are above or below the average. A trend is defined as a succession of seven points that are rising or falling. In a system without special causes for variation, a run or trend has approximately the same probability of occurring as a point outside a control limit, or .005. 7All cases of perioperative mortality, defined previously as death during or within two postprocedure days, 8,9which occurred at a suburban university hospital network between January 1, 1992 and December 31, 1994, were referred to the Department of Anesthesiology for peer review. Similarly, all cases of perioperative mortality, which occurred at an urban university hospital network between January 1, 1995 and December 31, 1999, were also referred for departmental peer review. A standardized model of peer review was used at both institutions under the author's supervision during these sequential time periods. 9Multiple referral sources were used, including the anesthesiologists, other clinical personnel using hospital incident reports, a follow-up phone call by nursing staff to ambulatory surgery patients, and concurrent chart reviewers. Contact was made with the anesthesiologist involved, and/or the medical record was reviewed, so that an abstract could be prepared by a preliminary group of two to four anesthesiologists for presentation to the respective departmental peer review committees. The peer review committees consisted of all available clinical members of the Departments of Anesthesiology for both suburban and urban hospital networks (approximately 25 staff anesthesiologists and 15 residents per committee for each). Each committee met on a monthly basis to participate in a structured peer review 9of the cases presented. The peer review process determined whether the mortality was solely the result of “system error” or whether there was a “human error” contribution. The American Society of Anesthesiologists (ASA) Physical Status of all patients undergoing anesthesia for operative procedures was only recorded at the suburban hospital network during 1994 and the urban hospital network from January 1, 1992 through December 31, 1999. Therefore, data collected during these years was used to extrapolate the distribution of ASA Physical Status over the two study populations. The Risk Management Departments of the respective institutions were also queried to determine whether legal action, previously defined as a letter of intent, claim, or closed claim, 10was initiated within 1 yr of any procedure that resulted in a perioperative death determined by peer review to be due, at least in part, to human error by an anesthesia practitioner.The principle underlying the peer review process is that all adverse outcomes are the result of “error,” either human or system. 9Nominal definitions for subcategorizing these two types of errors were used to add structure and increase the objectivity of the peer review process. 9,11,12Error here was defined as an act that through ignorance, deficiency, or accident departs from or fails to achieve a desired outcome. 13Human errors included failing to perform a technique properly, misuse of equipment, disregarding available data, failing to seek appropriate data, and responding incorrectly to available data due to a lack of knowledge. These human errors were considered deviations from the standard of care.System errors, on the other hand, result in adverse outcomes that might otherwise be considered unavoidable and ordinarily dropped from the peer review process. 14,15System errors included accidental occurrences resulting from performing a technique properly, equipment failure despite proper use, missed communication while following established protocol, inability to diagnose a disease process due to limitations of currently available screening and monitoring standards, inability to treat a disease process due to limitations in current standards of care, and inability to meet the demands for resources of equipment or personnel. Supervision by an attending anesthesiologist working with more than one resident or nurse anesthetist was viewed as a unique resource whose limitations were recorded separately from other resources. The peer review processes used by both Departments of Anesthesiology considered human errors on the part of nonanesthesia practitioners to be system errors if they were outside the control of the anesthesia provider. This would not prevent nonanesthesia practitioners from categorizing these adverse outcomes as human errors during their independent departmental reviews, but this process was unrelated to the anesthesiology peer review process. The error categories used by the Departments of Anesthesiology are summarized in tables 1 and 2along with common examples of each. 9,10Anesthesia-related mortality was defined as perioperative death to which human error on the part of the anesthesia provider, as defined by the peer review process, had contributed. If the departmental peer review committees determined that human error had contributed to an adverse outcome, they judged the degree to which the anesthesia care provider had contributed to that outcome. Contribution was graded on a three-point Likert scale ranging from minor to major. All determinations of the peer review committees were based on consensus.The Medline and HealthStar database queries identified 3,566 and 525 publications respectively. Review of the published abstracts that met inclusion criteria identified 23 anesthesia-related mortality rates determined between 1955 and 1992 by 21 different investigators. The results can generally be summarized in four major categories: (1) overall perioperative mortality ranged from 1 death in 53 anesthetics to 1 in 5,417 anesthetics, (2) anesthesia-related mortality ranged from 1 in 1,388 anesthetics to 1 in 85,708 anesthetics, (3) anesthesia considered solely responsible for perioperative death ranged from 1 in 6,795 anesthetics to 1 in 200,200 anesthetics, and (4) preventable anesthetic mortality ranged from 1 in 1,707 anesthetics to 1 in 48,748 anesthetics. Results of the literature review are summarized in table 3and plotted on an attribute P chart in figure 1.One hundred fifteen perioperative deaths occurred at the suburban university hospital network between January 1, 1992 and December 31, 1994. Two hundred thirty-two perioperative deaths occurred at the urban university hospital network between January 1, 1995 and December 31, 1999. Anesthetic caseloads were 37,924 for the suburban hospital network and 146,548 for the urban hospital network during those time frames. Calculated perioperative mortality rates were 30.3/10,000 anesthetics (1:332) and 15.8/10,000 (1:632) anesthetics respectively with an overall perioperative mortality of 18.9/10,000 anesthetics (1:532) as summarized in table 4. When broken down by ASA Physical Status the perioperative mortality rates in the suburban hospital network were 0 for 8,210 class I patients, 2 for 15,625 class II patients, 8 for 10,877 class III patients, 34 for 2,939 class IV patients, and 71 for 273 class V patients. Similarly, in the urban hospital network, perioperative mortality rates were 4 for 35,025 class I patients, 22 for 67,851 class II patients, 53 for 34,146 class III patients, 67 for 9086 class IV patients, and 86 for 440 class V patients. Perioperative mortality rates as a function of ASA Physical Status are shown in figure 2.Peer review determined that human error by an anesthesia practitioner contributed to 3 of the 115 (2.6%) perioperative deaths at the suburban university hospital network and 11 of the 232 (4.7%) perioperative deaths at the urban hospital network. Anesthesia-related mortality, defined as a perioperative death to which human error by an anesthesia practitioner contributed, occurred at a rate of 0.79 per 10,000 (1:12,641) anesthetics in the suburban setting and 0.75 per 10,000 (1:13,322) in the urban setting as summarized in table 4. When broken down by ASA Physical Status the anesthesia-related mortality rates in the suburban hospital network were 0 for 8,210 class I patients, 0 for 15,625 class II patients, 2 for 10,877 class III patients, 0 for 2,939 class IV patients, and 1 for 273 class V patients. Similarly, in the urban hospital network, anesthesia-related mortality rates were 0 for 35,025 class I patients, 3 for 67,851 class II patients, 2 for 34,146 class III patients, 6 for 9086 class IV patients, and 0 for 440 class V patients. Overall anesthesia-related mortality rate as a function of ASA Physical Status throughout the study period is shown in figure 3.Of the 14 anesthesia-related deaths overall, four were the result of major contributions from the anesthesia personnel (1:46,118 anesthetics), but only three resulted in legal action in the form of a letter of intent, claim, or closed claim (1:61,490 anesthetics). Of the three resulting in legal action, only one occurred in a patient with an ASA Physical Status of 1 or 2 (1:126,711 anesthetics in ASA Physical Status 1 to 2 patients).Our current original data suggest an overall perioperative mortality rate of approximately 1/500 anesthetics. This is consistent with the literature review, but the medical literature review offers a wide range of values. The wide range of perioperative mortality rates offered by the literature may be caused by differences in operational definitions and reporting sources. This is best illustrated by Pedersen (table 3), who described markedly different perioperative mortality rates in the same population depending on the timing of the patients’ deaths. 16Our current data are consistent with the perioperative mortality rate recorded by the JCAHO (approximately 1/300 anesthetics), which used the same definition and similar mandatory reporting for participating hospitals. ‡Our overall mortality rate (1:532) is also similar to the perioperative mortality rate reported by Vacanti et al. for 68,388 elective and emergency surgeries performed in 11 U.S. Naval Hospitals between 1964 and 1966 (1:25717). There are differences, however, when these perioperative mortality rates are stratified by ASA Physical Status. For example, Vacanti et al. 17reported a perioperative mortality rate of 1:1179 versus our rate of 1:10,809 for ASA Physical Status 1, and 1:11 versus our rate of 1:4.5 for ASA Physical Status 5. These differences may be caused by the small number of patients in both studies and our need to extrapolate the distribution of ASA Physical Status from a limited data set. Another possibility is differences in the application of ASA Physical Status designations. For example, by definition, ASA Physical Status 5 patients are not expected to survive 24 h with or without their planned operative procedure, thus our higher mortality figures for this class of patients seem more credible. It is also possible that not all of the perioperative deaths were captured. Our use of mandatory reporting to a nonpunitive peer review process, multiple reporting sources, and the severity of the outcome assure a high capture rate for this occurrence 18and may also account for some of the difference between our data and that of Vacanti. Vacanti's study was excluded from our literature review because it did not include an anesthesia-related mortality rate.Twenty-three anesthesia-related mortality rates were identified, however, by our literature review. Wide variation in anesthesia-related mortality rates reported over the last 35 years may also be due to differences in operational definitions. Definitions varied from intraoperative deaths 19–22to deaths occurring within 30 days 23or prior to discharge from the hospital. 16In some studies, perioperative “death” also included patients who failed to regain consciousness after anesthetic management, 24–26or who died any time during their hospital stay following an intraoperative cardiac arrest. 27This author used the 1992 JCAHO (Joint Commission on Accreditation of Healthcare Organizations) definition of perioperative mortality. “Death of patients during or within two postprocedure days” was one of the JCAHO Perioperative Performance Indicators that survived α and β testing for face validity and feasibility of data collection in a broad range of healthcare institutions. 28Anesthesia-related mortality was defined as perioperative death to which human error on the part of the anesthesia provider, as defined by our peer review process, had contributed.The principle underlying our peer review process is that all adverse outcomes are the result of “error” defined as an act that through ignorance, deficiency, or accident departs from or fails to achieve a desired outcome. 13This definition of error is consistent with the IOM definition:“Failure of a planned action to be completed as intended” or “use of a wrong plan to achieve an aim; the accumulation of errors results in accidents.”1Both of these definitions allow reviewers to look at the system as critically as they look at each other, thus making peer review less threatening. Katz and Lagasse have shown that anesthesiologists will comply with a system of self-reporting if the process is nonpunitive and can result in real improvements in patient care. 18In addition to its effect on self-reporting rates, peer review can also affect published anesthesia-related mortality rates through the accuracy of its judgments.Although the accuracy of judgments by a peer group can never be assured, interrater reliability can give some indication of the reproducibility of the data. Several measures have been taken to improve the reliability of our peer review process. Use of multiple reviewers who meet to discuss the case has been shown to markedly increase consensus among reviewers. 11,12,29,30During the course of this study, the faculty of the departments of anesthesiology remained relatively constant so that the members of the peer review groups remained stable. Structured assessment procedures have also been recommended to decrease differences in reviewers’ understanding of their task and thus to increase the objectivity of implicit peer review. 11,12By using nominal definitions for categorizing peer review opinions regarding adverse outcomes, the error analysis was relatively easy to apply so that the errors could be reliably identified and grouped. Structured peer review and a stable pool of reviewers allow the error categories to become more sharply defined over time. 9Shared expertise in a particular area also improves agreement among reviewers. 31All of our reviewers were anesthesiologists or resident anesthesiologists as defined by the composition of each department. Although some investigators have suggested that outcome data be withheld when determining appropriateness of care, 32others have suggested that outcome data are necessary to assure adequate agreement among multiple reviewers. 11A recent study of structured peer review models showed no relation between severe outcomes and subsequent classification as human error. 12This study also showed that the peer review model used in the current study has excellent interrater reliability when used in the manner described.As pointed out by Keats 33more than two decades ago, high interrater reliability in a peer review process does not assure that judgments are accurate. Judgments are shaped by the knowledge, experience, and current norms of the reviewers, and therefore, may vary over time. Consequently, to make valid comparisons with historical controls, outcomes databases must record the circumstances surrounding each event in addition to the judgments rendered. This allows subsequent peer review at a later date and removes the of an peer review process. Although the author's database abstracts of each event for subsequent review at a one can a more database that is to an of data recorded by a clinical anesthesia management system for similar current study defined anesthesia-related mortality as perioperative death to which human error on the part of the anesthesia provider, as defined by our peer review process, had contributed. Although the review mechanisms this was the definition used in the studies reviewed for this (table all however, to this for example, reported an mortality rate that was to perioperative mortality rate of anesthetics, but on to report a preventable anesthetic mortality of anesthetics. anesthesia mortality was defined by and other as death of patients in care on the part of the anesthesia provider was to have resulted in patient mortality rate that was reported in the literature reviewed was that in which anesthesia was considered solely This mortality rate was defined as patients a perioperative death as a result of human error on the part of the anesthetist perioperative mortality as a result of the anesthetics patients in anesthesia-related deaths were to be due to patient of anesthesia-related deaths to deaths in which anesthesia was considered solely responsible may have led the IOM to that anesthesia mortality rates have decreased from 2 deaths per 10,000 anesthetics administered in the 1980s to about 1 death per 200,000 to 300,000 anesthetics administered one in our literature review reported an anesthesia mortality rate of less than 1 in 200,000 anesthetics. study, intraoperative deaths in ASA Physical Status 1 and 2 patients, which were reported to the between and were considered to be solely to this time period, anesthetics were administered to ASA Physical Status 1 and 2 patients, thus a rate of death solely to anesthesia of 1 per 200,200 one be that the data the of the claim by the IOM that anesthesia mortality rates have decreased by an of over the two it is to these into by with our current original data. The study on data reported to a the current data at human error as determined by peer review. A recent study showed that risk as defined by a letter of intent, claim, or closed claim has no to human errors by anesthesiologists that result in patient as determined by peer review. study only considered intraoperative deaths that occurred in patients with an ASA Physical Status of 1 to 2 in which anesthesia was considered the the 14 anesthesia-related deaths reported in the current original data, only 4 were the result of major contributions from the anesthesia or 1 per anesthetics. Of our peer review process has never considered a death to be due solely to anesthetic disease or other have been to be Of the four deaths with major contributions by the only one occurred in a patient with an ASA Physical Status of 1 to 2 and resulted in over the period in which anesthetics were performed in these of patients. this single death did not and would not have met the inclusion criteria of the Therefore, our are consistent with the study, which may have been the basis of the IOM our literature some on the in anesthesia safety, in of anesthesia related mortality, there are some it fails to anesthesia-related mortality reported prior to 1966 because of the limitations of the Medline In and reviewed anesthetics administered in institutions and reported a perioperative mortality rate of 1 in These investigators reported that anesthesia was the of mortality in 1 in cases and was in 1 in and reported a of deaths in anesthetics administered between and was attributed solely to anesthesia in 1 in anesthetics and in 1 in patients. et at the of reported perioperative deaths within 30 days of the approximately anesthetics performed between and was considered a to death in 1 in anesthetics and 1 in general anesthetics. this report by et al. no deaths in any of the patients with ASA Physical Status Although these studies suggest improvements in anesthesia safety over the years, they have not been included in our control chart because it is to assure of all studies the Medline database a lack of appropriate risk it to in anesthesia safety because study may both and with to perioperative For example, in and that anesthetic procedures had by a similar was in When for the increase in the showed that the rate of anesthetic procedures had from to per More patients and those with higher ASA Physical Status showed the In between and the number of anesthetic procedures for patients with an ASA Physical Status of 1 had by while the number of procedures for patients with an ASA Physical Status of 3 had by this trend were it could improvements in anesthesia trend concurrent disease could also make the author's use of to the distribution of ASA Physical Status for the current original data less than In the current study, that could have resulted in an of patient disease because ASA Physical Status was in the of each study period and and to the of ASA Physical Status might then have resulted in a mortality rate that is than than the of the of ASA Physical Status between the urban and suburban hospital networks a higher of ASA Physical Status 4 and 5 patients this is more to be due to differences than a trend in perioperative addition to patient differences may also variation in practice standards, resources, and reporting These might be when the differences different This may have resulted in in the anesthesia-related mortality rates in our process control chart and in to support this can be in the anesthesia-related mortality studies in a single For example, in New there to be in anesthesia-related mortality. a in anesthesia-related mortality from approximately in to in and in Similarly, a in anesthesia-related mortality over a period at a hospital in these single studies not dramatic improvements in anesthesia-related mortality rates over the three in safety have been in other that on The safety of for has in this but the there has been in may be due to the effect that improved safety has had on has made it possible to meet of the by more and with less between this improvements in This may be to the practice of anesthesiology in which improvements in medical have led to anesthetic management of patients with more concurrent have shown 1 and the risk of death in these patients as does the risk of death in which human error by an anesthesiologist is is not that our control charts did not a trend in anesthesia-related mortality over time. The of our data points are more than three standard deviations from the anesthesia-related mortality of deaths per 10,000 anesthetics, which anesthesia-related deaths that occurred in anesthetics reported in the anesthesia-related mortality is not a stable system. A system that is does not have a In other one in anesthesia safety in of anesthesia-related mortality special causes of variation have been These special causes of variation may real differences in anesthesia safety between the various or differences in the used to anesthesia-related mortality current data that the overall perioperative mortality rate for patients ASA Physical Status is approximately 1 per anesthetics. The literature suggest a wide range of perioperative mortality rates, which are caused by differences in operational definitions and reporting sources, as as a lack of appropriate risk Our current data, however, are consistent with reported perioperative mortality rates using the same definition and similar mandatory reporting for participating hospitals. Our data suggest that the anesthesia-related mortality rate, as determined by peer review, has been stable over the last at approximately 1 death per anesthetics. Wide based on differences reported in the literature make it to in anesthesia on these the are It is time to the that is not any must the that anesthesia-related mortality has improved by an of does not support this must then our to our methodology of data collection and analysis so that can data data will allow to risk models and identify best then can anesthesia become a model of