Governmental Databases, Hospital Information Systems, and Clinical Outcomes: Big Brother or Big Help?

June 21, 1999. The past decade has seen an intense interest in the development of large scale governmental quality assurance databases to track clinical practices and outcomes (1,2). Cardiac surgery (coronary artery bypass surgery), as a result of the large number of similar procedures performed, robust clinical endpoints (e.g., myocardial infarction, death, and stroke), and economic impact, has been the specialty most studied by government. New York State has been a leader in this area with the establishment of the Cardiac Surgery Reporting System (NYCSRS) in 1989 (1). Although the results of the clinical practice of individual surgeons are reported annually, patterns of patient care and, in particular, anesthetic management, have never been published. In a landmark study, Reich et al. (3) have linked data from the NYCSRS to the intraoperative records of patients operated on in two New York City hospitals. Using these two databases, the investigators assessed the impact of intraoperative hemodynamics on perioperative outcomes. Before determining the importance of their findings, it is critical to understand the benefits and limitations of using these large-scale administrative databases. From an evidence-based medicine perspective, these data are important for generating hypotheses but not cause and effect relationships (4). In other words, the data itself are not specifically collected to address a clinical question, but rather represent a quality assurance process. For example, the primary use of the NYCSRS has been to benchmark outcomes between institutions and individual surgeons. Because patient demographics (i.e., risk factors) may differ dramatically between surgeons and institutions, it is important to risk adjust these outcomes. The factors which are incorporated into any risk adjustment have been evolving over the last 10 years, and the optimal system has yet to be developed (5). For the period of the study, the NYCSRS risk factors used in the adjustment models include age, gender, hemodynamic state, severity of atherosclerotic process, ventricular function, renal failure, and previous heart operations (6). Although data collectors are trained, this database is self-reporting and, therefore, may include errors and omissions. There have been numerous concerns that upcoding of comorbidity has occurred as a means of reducing an individual surgeon’s or institution’s risk adjusted mortality. Additionally, the outcomes of interest require more precise definitions, and these, unfortunately, can still vary among institutions. Since 1992, there have been annual audits of a sample of the data from a sample of the hospitals, with no demonstrable difference in expected mortality rates from the original compared with the audited data, thus supporting the validity of the data. Although mortality is a well defined endpoint, perioperative myocardial infarction or stroke is highly dependent on the surveillance protocols and physician interpretation. Parallel to the development of governmental databases has been the implementation of hospital information systems and continuing refinement of intraoperative data record keepers. In either system, the data points captured on individual patients are aggregated and linked to predefined clinical and/or resource utilization outcomes. In addition to issues related to data collection and risk adjustment in the NYCSRS, factors critical to accepting the results of the study of Reich et al. (3) include the accuracy of the information obtained from the operating room record keepers. Algorithms that allow editing and potential “smoothing” of intraoperative data may add to the inaccuracy or introduce bias, rather than improve it. However, studies in this area have reported a higher degree of accuracy than data obtained from a traditional “hand written” record or self-reporting (7,8). Of the two data collection streams that form the basis of this report, the operating room information is probably more accurate. This study is unique in one other important aspect: the data are reported separately from two institutions in two different healthcare systems, rather than the traditional single site or aggregated multi-center reports. Is there information we can learn by comparing two sites? Simply put, yes! Both the similarities and differences are important not only to the clinician but also the healthcare researcher. As a means of highlighting this information, we have segregated the predictors of mortality by significance of association in each site in Table 1. The fact that the “hard” outcome, mortality rate, is low and similar (2.89%) between the two sites provides reassurance that overall care is equivalent. The large discrepancy in the incidence of perioperative myocardial infarction (5.4% versus 1.0%) after cardiac surgery could be the result of two potential mechanisms. Of course, there could truly be differences in the rate of perioperative myocardial infarction between the two sites, although the fact that rates of other outcomes are similar suggests that this may not be the case. More importantly, it is likely that the threshold for diagnosing a perioperative myocardial infarction in the cardiac surgery database is different between the two sites. The diagnostic criteria for this diagnosis can vary greatly for both clinical and research purposes (9). Table 1: Predictors of Perioperative Mortality By InstitutionAlthough the risk-adjusted mortality rate for both sites are essentially similar for the study years (2.89 for Mount Sinai compared to 2.89 for St. Luke’s Roosevelt), there are differences in the observed mortality rate (6). Mount Sinai demonstrated an observed mortality of 3.37 with an expected of 3.00, while St. Luke’s Roosevelt had an observed mortality of 2.64 with an expected mortality of 2.35. This suggests that there are subtle differences between the two sites with respect to patient populations. Examining the risk factors with respect to their association with mortality at each of the sites (Table 1) reveals that some of the risk factors in the final multivariate model were not found to be important predictors for each site. This suggests that there may be important differences in perioperative care which influence mortality at each site and that the aggregate answer may not be generalizable. For example, age, diabetes mellitus requiring medication, angioplasty before current admission, and old myocardial infarction were significant risk factors for mortality at only one site. These risk factors are all important for risk adjustment for the NYCSRS and for other models, which suggests that either care for these patients are different or the populations in the two hospitals are different. Does this mean that we should pool data? We can, but we should understand the limitations. Second, should this information be used for institution or physician profiling? As discussed above, the two centers had similar aggregated risk-adjusted mortality rates for the 3-yr period of the study, but actually had different risk-adjusted mortality rates (2.89 for Mount Sinai compared with 4.33 for St. Luke’s Roosevelt) during the final year of the study (1995) (6). They also demonstrated different perioperative processes that affected mortality (Table 1). Before using this information to make decisions regarding the appropriateness of care, it is important to acknowledge inherent limitations in these techniques. As demonstrated by Hofer et al. (10) regarding diabetes care, physician variation in practice may actually account for less than 4% of resource utilization and outcome if more complete risk-adjustment is performed. Although the report cards on cardiac surgery have been available in many states for a number of years, consumers do not seem to rely on this information to make decisions regarding their providers (11). Therefore, the ultimate value of these databases remains undefined. Finally, it is important to look at the actual clinical findings of this study in the light of previous research on hemodynamics and cardiac morbidity. Several single center studies have reported the relationship between hemodynamics and prebypass myocardial ischemia. Urban et al. (12) and Leung et al. (13) were unable to find a strong association between prebypass hemodynamic alterations and myocardial ischemia. Specifically, Urban et al. (12) reported a positive predictive value of 24% or less for hemodynamic variable preceding ischemia, although Leung et al. (13) reported that only 28% of the episodes of ischemia were associated with hemodynamic abnormalities. Yet the current study found a strong association between perioperative hemodynamics and “final” outcomes. Does this suggest that one study is correct and the other study is incorrect? It is important to evaluate closely what each study is measuring. Myocardial ischemia is a process variable that has been associated with myocardial infarction (14). In fact, many more patients become ischemic than actually sustain irreversible myocardial necrosis. Because the development of myocardial ischemia is multifactoral, it is conceivable that the hemodynamic alterations that can lead to myocardial ischemia also lead to myocardial infarction, although nonhemodynamic related myocardial ischemia may not lead to irreversible events. Additionally, the predictor prebypass low mean arterial pressure was significantly associated with mortality in only one of the sites. This finding suggests it may be related to some other factors unique to the surgical or anesthetic technique or even the patient population itself. Another interesting finding relates to tachycardia. Since the original work by Slogoff and Keats (15), prebypass tachycardia has been considered one of the primary determinants of myocardial ischemia and infarction. In the current study, prebypass very high heart rate was associated with mortality at only one of the sites, was not predictive of mortality in the multivariate model, and only predicted myocardial infarction with a relatively small odds ratio (heart rate high, odds ratio 2.0). Does this mean prebypass tachycardia is not bad? Of course not. Rather, it is likely that prebypass heart rate is generally well controlled and that only rarely patients have this severe hemodynamic abnormality. Factors which occur at a low frequency will not be included in the model. Interestingly, postbypass very high heart rate was associated with a high risk (odds ratio 3.1) for overall mortality. Control of heart rate during this period may not always be a primary objective, but the current study suggests that this is probably an important goal even after bypass. The study confirmed other strong predictors (odds ratio >3.0), including preoperative hemodynamic instability, calcified ascending aorta, and preoperative renal failure, all of which are established risk factors for adverse outcome after cardiac surgery. There remain many unanswered questions regarding the optimal care of patients undergoing coronary artery bypass surgery. The paper by Reich et al. (3) offers important insights into an elegant method in which future research can be undertaken to address these questions.