A Longitudinal Study of Clinical Peer Reviewʼs Impact on Quality and Safety in U.S. Hospitals

EXECUTIVE SUMMARY Clinical peer review is the dominant method of event analysis in U.S. hospitals. It is pivotal to medical staff efforts to improve quality and safety, yet the quality assurance process model that has prevailed for the past 30 years evokes fear and is fundamentally antithetical to a culture of safety. Two prior national studies characterized a quality improvement model that corrects this dysfunction but failed to demonstrate progress toward its adoption despite a high rate of program change between 2007 and 2009. This study's online survey of 470 organizations participating in either of the prior studies further assessed relationships between clinical peer review program factors, including the degree of conformance to the quality improvement model (the QI model score), and subjectively measured program impact variables. Among the 300 hospitals (64%) that responded, the median QI model score was only 60 on a 100-point scale. Scores increased somewhat for the 2007 cohort (mean pair-wise difference of 5.9 [2-10]), but not for the 2009 cohort. The QI model is expanded as the result of the finding that self-reporting of adverse events, near misses, and hazardous conditions—an essential practice in high-reliability organizations—is no longer rare in hospitals. Self-reporting and the quality of case review are additional multivariate predictors of the perceived ongoing impact of clinical peer review on quality and safety, medical staff perceptions of the program, and medical staff engagement in quality and safety initiatives. Hospital leaders and trustees who seek to improve patient outcomes should facilitate the adoption of this best practice model for clinical peer review. INTRODUCTION Clinical peer review has long been pivotal to physicians' efforts to improve quality and safety. Considering its importance to the profession and society, surprisingly little research has been directed toward optimizing the process. In the United States, the prevailing process model—quality assurance (QA)—emerged as an unintended consequence of Joint Commission standards promulgated in 1979 that called for an organized program of QA (Sanazaro & Mills, 1991). In order to comply, hospitals abruptly abandoned the clinical audit model, which had previously been promoted by professional standards review organizations. The resulting QA model has been widely criticized as being at best ineffective and at worst antithetical to true quality improvement (Berwick, 1990; Dans, 1993). By focusing narrowly on questions of individual competence to meet the standard of care, the QA model evokes fear of censure and loss of livelihood. It thereby contributes to the persistent culture of blame documented in the 6-year trend in Survey of Hospital Safety Culture data, in which the composite measure “nonpunitive response to error” has run consistently at an abysmal 44% positive (AHRQ, 2012). The associated unstructured methods of evaluation have low reliability (Goldman, 1994). Moreover, the use of generic screens for adverse events to identify cases for review is inefficient, particularly compared to the selfreporting process adopted by airlines and other types of high-reliability organizations (Sanazaro & Mills, 1991; O'Neil et al., 1993; Helmreich, 2000). A 2007 study was the first to report normative data about clinical peer review practices on a national scale (Edwards & Benjamin, 2009). Using a high-level process framework (see Figure 1), it collected data on 39 items from 339 institutions including 61 major teaching hospitals. The study identified wide variation in the scope of activities encompassed by peer review programs beyond the essential core process of retrospective medical record review (case review).FIGURE 1 Clinical Peer Review Process FrameworkThe findings from this study suggest that medical staff peer review may be the dominant method of adverse event analysis in U.S. hospitals. The median case review volume (1-2% of hospital inpatient volume) is an order of magnitude closer to the known rate of preventable adverse events than is formal root cause analysis, the mandated method for the most serious occurrences. A high rate of program change was observed due to dissatisfaction with the QA model and changes in Joint Commission requirements. Most importantly, the 2007 study revealed a set of practices that are strongly associated with the belief that the program has a significant ongoing impact on the quality and safety of care. Such practices include standardization of process, recognition of clinical excellence, attentive program governance, trustee involvement, integration with other hospital quality improvement activity, timely performance feedback, and identification of clinician-to-clinician issues and other process problems during case review (Edwards & Benjamin, 2009). These practices define a new approach to clinical peer review best described as the quality improvement (QI) model. The QI model contrasts sharply with the QA model (see Table 1) and can be understood as the product of consistent application of quality improvement methods to peer review program process, structure, and governance. It highlights the extent to which the QA model has disconnected the ends and means of peer review activity.TABLE 1: Comparison of QA and QI Models for Clinical Peer ReviewIn 2009, the QI model was validated in a separate cohort of hospitals against both subjective and objective measures of quality and safety (Edwards, 2010, 2011). The validation study demonstrated that important differences among clinical peer review programs can predict up to 18% of variation on 32 standardized performance measures. It also suggested that organizational factors, such as leadership and openness to change, influence program effectiveness. In addition, it showed a persistently high rate of change, wherein many programs were shifting from clinical depart-ment-based review to multispecialty committee-based review. However, the 2009 study yielded no clear evidence of advancement of the QI model. These studies raised important new questions. Does the QI model better serve to engage physicians in quality and safety? Has any benefit been derived from the trend toward the multispecialty committee process? Is self-reporting of adverse events being encouraged in hospitals? If so, what is its influence on peer review program outcomes? The present study was undertaken to answer these questions and to further assess the evolution of peer review practices and their impact on quality and safety. The above questions acquire added importance in the context of theories of quality improvement and patient safety. The pioneering work of Juran (1989), Deming (1982), Crosby (1979), and others launched the heyday of total quality management in the 1980s and contributed to the concept of the learning organization (Senge, 1990). While the methods and tools for quality improvement have continued to proliferate, all share a focus on business processes in the context of the organizational system in which they occur. They also emphasize the essential responsibility of leadership to shape organizational culture and performance by communicating the vision and the performance standard, organizing for quality improvement, and providing adequate supports (attention, resources, tools, and training). Defects are most often a property of the system and are infrequently the fault of individual workers. Therefore, the blame commonly cast on individuals found at the “sharp end” of an adverse event is demoralizing to staff, leaving the organization powerless to improve performance. More recently, attention has been given to identifying the factors that have fostered high reliability and safety in complex, dynamic, high-risk environments (Reason, 2000; Weick & Sutcliffe, 2001). Healthcare leaders have begun to champion high reliability as a worthwhile goal, if not a mandate (Chassin & Loeb, 2011; Denham et al., 2009). High reliability demands a culture of safety characterized by trust; the imperative to report and improve; and the collective mindfulness of the high cost of process failure, the inevitability of human error, and the need to identify, contain, and recover from errors as early as possible. It combines the systematic pursuit of quality improvement with constant vigilance for the unexpected. The skill of managing the unexpected well is underdeveloped in most organizations, largely because it requires a counterintuitive act: a strong response to a weak signal (Weick & Sutcliffe, 2001). METHODS The present survey was conducted under the auspices of a patient safety organization listed by the Agency for Healthcare Research and Quality. The sample was constructed from the records of the two prior national studies cited earlier. The 2007 study was sponsored by the American College of Physician Executives (ACPE); the University HealthSystem Consortium; Premier Inc.; and hospital associations in Arkansas, California, Florida, Michigan, Missouri, South Carolina, and Wisconsin. Each organization used its own process for inviting those with working knowledge of their hospital's clinical peer review program to participate in an anonymous online survey. On the basis of voluntarily provided e-mail and/or phone contact information, 158 organizations participating in the 2007 study could be positively identified retrospectively. One of these organizations subsequently merged with another participating facility. The ACPE alone sponsored the 2009 study. From 362 complete responses, 330 unique organizations were identified, two of which have since closed; 15 had also participated in the 2007 study. Thus, the combined data sets yielded 470 unique organizations broadly representative of U.S. hospitals. The survey instrument incorporated the QI model and those items previously used that had the greatest expected value for comparative purposes. It also collected information on additional factors potentially related to program effectiveness, such as the perceived quality of case review. Several physician leaders assessed the draft for clarity and completeness. The survey is reproduced in the online supplement to this article (www.ache.org/Publications/SubscriptionPurchase.aspx). The degree of conformance to the QI model (the QI model score) was calculated on a 100-point scale on the basis of the responses to specific survey items according to the method previously developed (Edwards & Benjamin, 2009) (illustrated in the online supplement). Key program outcome variables included the following: Study participants' perceptions of their program's ongoing impact on quality and safety Medical staff's perceptions of the peerreview process Physician engagement in quality and safety improvement Overall physician-hospital relations. A total of 40 items were captured via four web-based forms. Prior participants received e-mail solicitations. Phone calls were made to nonresponding organizations to identify alternative contacts. Data were collected from September 15, 2011, through January 26, 2012, as patient safety work product under the terms of the federal Patient Safety and Quality Improvement Act of 2005. In accord with its statutory protections, only aggregate, nonidentifiable data are disclosed in this article. A response was considered complete if all four pages of the survey were submitted and partial if two or three pages were submitted. A response with fewer than two pages was considered a breakoff. Both complete and partial responses were included in the analyses. In the few situations in which multiple responses were received from a given facility, one set of responses was chosen according to organizational titles of respondents. The longitudinal change in QI model scores was characterized by the mean difference using a two-sample t-test. Ordinal logistical regression was used to define the primary factors contributing to program impact on quality and safety, medical staff perceptions, medical staff engagement, physician-hospital relations, and reviewer participation. Alternative models were developed that either allowed for or excluded other outcome variables as predictors. For comparative purposes, at least one model was developed for each outcome that included the QI model score as a predictor. Outlier values were retained. A regression equation was accepted only if all the factor coefficients and intercepts were significant at p < .05 and if the goodness-of-fit test met p > .1. Response levels were selectively collapsed as guided by this goal. The estimate of variance explained by each model was taken from the equivalent linear regression model R2. Statistical analysis was carried out using Minitab version 15 (2007). RESULTS The survey process yielded 297 complete responses, three partial responses, and four The response rate was Among these were and hospitals. The were either leaders or The were were no significant differences between and hospitals on the basis of prior survey QI model perceived quality and medical staff in the online the responses for survey items that clinical peer review to be a of hospital quality and safety improvement it is that the program a significant ongoing to quality and safety, and report that it at least on a scale of importance to all QI of not their case review evidence of of attention to program was present in the prior Table highlights changes in the scope of activities encompassed by peer review programs the past The change has been to physician program to the but which was not previously is included by Both of these activities to Joint Commission of hospitals separate the peer review program from the process but use the of peer review in including ongoing professional practice evaluation in Clinical Peer Review 2007 to the used to identify cases for peer generic screens including physician or hospital staff and from or clinical are professional practice evaluation for new is included the scope of the peer review program at of it infrequently to the case review process. A of and methods are used to these but it to on by or integration with quality improvement so, at 15 study self-reporting is among the three for case with leaders at of the 15 hospitals revealed that this was through Moreover, of or strongly that staff report adverse events, near and/or hazardous their own for peer Table changes in data captured during case review. of programs the of More than record additional data including of quality of by individual one findings of in care. A was in the identification of which is associated with the QI model, but also in the use of which are associated with the QA in Data Review 2007 to two of programs and one mandate as a for or to new This most commonly review and and programs also to review methods and or management and QI methods are not commonly was the case in only one in programs peer review in a committee In about two of these the committee has multispecialty physician In 2009, no programs from or other In the present hospitals that leaders or staff are While these are in terms of they not a peer review process in which all each performance on an for other factors, multispecialty committee process has no significant to program effectiveness. is a significant of the perceived impact on quality and safety, with an of for a to the of committee The trend since 2009 has been for of programs to major in program structure, process, and/or each These changes have commonly multispecialty or of the process. QI model scores from 1 to with a median of A in the score was associated with an in quality and safety impact with an of of programs a that to the QI model. For the 2007 QI model scores The mean difference was 5.9 quality impact and medical staff perceptions of the program also For the 2009 a in perceived quality impact was but no change in QI model scores or medical staff perceptions was Table a of regression models program and organizational variables to The QI model score a factor impact on quality and medical staff Self-reporting and the quality of case review in regression models that for of the variation in the perceived ongoing program impact on quality and safety, of the variation in medical staff perceptions of the program, and of the variation in physician engagement in quality and safety. A to the of self-reporting is associated with a of quality Medical staff perceptions are physicians are and the program the QI model. in Table of the variance in reviewer is explained by medical staff perceptions of the program, the extent to which leadership is to any program change, the quality of case and the degree of process of Key in Models for Clinical Peer Review This study is the first to report longitudinal data on clinical peer review practices in the United it of the factors contributing to program the QI model to include self-reporting and the quality of case the of openness to change, and to a on the value of multispecialty committee and important between the peer review process, the perceived program impact on the quality and safety of care, physician engagement in quality and safety, and physician-hospital relations. The study for patient quality and safety that need the attention of all hospital not physician the high rate of program change, hospitals have been to the QI model. A in QI model scores was observed for the 2007 cohort. This finding the of the years than differences between for improvement for than of The QA model the culture of which to U.S. hospitals and progress in a culture of safety, the QI model to organizational learning from in the system of care. Thus, hospital leaders and trustees should be about the that result from of the This study the and importance of self-reporting of adverse events, hazardous and near by of the medical the in that have the of high reliability strongly to the value of only one program was previously known to have high of self-reporting of cases of events by all and clinical outcomes & 2000). this have evidence that a of hospitals have not only encouraged the practice but are the expected improvement in quality and safety. Self-reporting can only in the of a organizational from for self-reporting (Edwards, 2012). Self-reporting also requires leadership clear about the program, of a review process, and resulting to self-reporting the management of these and thereby progress toward peer review program and a culture of safety. While this study on the are to and other of their performance is by or Self-reporting also the in By generic screens their are but not They are to and many Self-reporting can this the past few hospital in the United have been by the concept of physician engagement & The Act has a of in primary physician practice management as a means of that engagement and the associated for cost & 2011). Physician engagement in quality and safety is by perceived peer review program impact on quality and safety, medical staff perceptions of the program, and physician-hospital of In if what factors most to the program impact on quality and safety, that about the variation can be explained by of the QI model physician engagement, the quality of case and the program's importance to all other QI at the These are to physician and hospital They should be in of the specific for From a management if the organization not have strong physician engagement, the QI model peer review process and self-reporting that have it the other of to the QI model by all that supports quality and safety. This study also considered physician-hospital as a program that peer review is only one factor among many with so, with medical staff who in the peer review process with this objective in By responsibility to a result of value to the the physician leadership for hospital management to with changes that were important to can be by a of The regression model for physician-hospital medical staff perceptions of the peer review program, physician engagement in quality and safety, and openness to change, of the The QI model score of the While the between peer review and physician-hospital is than that with quality and safety, clinical peer review program improvement can be In Joint Commission for professional practice evaluation and dissatisfaction with the QA model have been the primary of peer review program committee-based review has been as a The concept has the of review. The of other may the of which is essential for the pursuit of high reliability and in the system and process issues that to adverse It is by to the dysfunction of the QA model and the culture of blame to an of the benefit for quality and safety improvement from process standardization and the use of committee In it is the that the learning and that is to particularly the process is and in clinical performance. of most case review committee In physicians be to participate as peer the program is well for contributing to in quality and safety, the process is the quality of case review is and leadership is is the program has a and but they if it is perceived as This study has It on a at each validation of responses was Survey and response the three Quality and safety impact be assessed with objective performance measures or by such as the of improvement in to patient Hospital may have been in their for medical staff and physician-hospital relations, which be measured through survey of medical If so, the should be strongly with that standard and of value in regression The other issues are by the of the observed multivariate their and their through the three The high response rate the for The of difference in prior survey data for and further in this One can that the clinical peer review process has to improve in most hospitals. It all for it to be conducted to performance and Such practice is an management has the in hospitals. leaders quality improvement They in quality of and patient safety. They and improve all core Thus, they should no longer a to peer review. need measures of program performance to the QI model While a measure of preventable be no is widely Hospital event only a of adverse events with tools significant physician review that be better directed toward identifying improvement through the QI model. such as the hospital standardized are and not the preventable In any the only is preventable et al., which the system is of self-reporting is the because the be made the be through case review and other and all and leaders be in the In the the best measures may be of improvement identified, and individuals for performance in to total cases with the of cases to hospital The QI model strongly supports a culture of safety and has the to improve patient The can be by self-reporting and by the primary during case review from the standard of to can to improve clinical Hospital leaders and trustees should the medical staff to peer review and adequate to change improvement should not be The the of and who assessed the draft survey for clarity and and of and who the draft