Answers are grounded in this paper and cite their sources.
National collaborative audits are essential for accurate risk stratification and performance monitoring in high-risk vascular surgery.
The current status of audit and clinical governance can be likened to a fibrillating heart: lots of energetic and ineffective activity with no useful output. A recent study involving 47 NHS trusts in the West Midlands concluded that while clinical governance has led to the development of strategies, establishment of committees and appointment of ‘leads’ in clinical governance, there was no evidence of the required cultural change or that clinical governance has yet to make any real difference [1]. Part of the problem lies with the term itself, and the lack of any clear definition that might focus our attentions. However, one approach that shows some promise is the measurement of the outcome of treatments or interventions by individual units and comparison with outcomes measured for similar interventions across the whole country. Getting reliable data from multiple units while accurately stratifying for risk is a feasible but thus far elusive goal [2, 3]. The mortality from elective aortic aneurysm repair has for years been quoted by our surgical colleagues as being ‘around 5%’, a figure largely derived from published series [4, 5]. It is intuitively obvious that many factors will affect that figure, including patient age and comorbidity, and the level of excellence displayed by each individual unit. It is also clear that while some units may claim an overall mortality of less than 5%, many anaesthetists believe that this figure is optimistic. It is reasonable to expect authors to publish their results if they appear to be good; no one is likely to publish figures suggesting that their results are poor. Thus, published series are likely to underestimate overall mortality, and they do not adequately address the issue of differing case mix. To attempt to define what the true mortality from elective aortic surgery is throughout the UK, and to determine the effect of known or suspected risk factors on the mortality, the Vascular Anaesthesia Society of Great Britain and Ireland (VASGBI) commissioned a multicentre audit, the findings of which have been recently published [6]. The outcome measure chosen was in-hospital mortality, which was felt to be more useful than 30-day mortality, as the latter would be likely to miss patients dying after a complicated and prolonged postoperative period. Elective and urgent operations were included, but not emergency operations. Aneurysm repair and operations for aorto-occlusive disease were included, but analysed separately where appropriate. Data on a total of 933 patients were gathered over a 4-month period. Overall mortality was 7.3%, but this was markedly affected by risk factors. For example, whereas mortality in those patients aged < 65 years was 4.4%, it was 12.1% for patients aged 75 years. There are several benefits deriving from the performance of this audit. The first is the provision of a reliable standard against which individual units can measure themselves. It is to be hoped that all units performing high-risk surgery are already collecting data on the patients, operations and outcomes; now their results can be put into context. Second, the provision of these data will allow more accurate and informed discussion with the patient regarding the risks of proposed surgery. Third, new information has emerged that may need testing by further observational studies. Examples include the fact that previous myocardial infarction (more than 3 months before surgery) does not increase mortality from aortic surgery, and that the presence of two or more ventricular extrasystoles in the pre-operative electrocardiogram is associated with a nearly three-fold increase in mortality. So what now? It appears that the anaesthetic community in the UK is capable of collaborating to produce large-scale multicentre audit, and each individual contributing anaesthetist is to be congratulated. However, we should not be content to stop there. Audit of this type needs to continue, with the aim of developing a registry of all procedures of a particular type being done. This will allow ever-increasing refinement of the risk-stratification model, monitoring of performance of individual units, and will encourage the publication of powerful observational studies. Mercifully, we anaesthetists are not alone. The Vascular Surgical Society of Great Britain and Ireland (VSS) has been working for several years on its own version of a National Outcomes Audit, which aims to collect data to allow accurate risk stratification and monitoring of performance, and so to encourage maintenance of standards across the country. The advantage of co-operation for us is that the surgical societies have greater access to the facilities required, and techniques and software are often already in place for the collection of large amounts of data. The advantages to the surgeons of having anaesthetists as partners include the fact that this co-operation reinforces the notion that the results being monitored are the result of teamwork, rather than being dependent upon the dexterity of the surgeon. In addition, we anaesthetists are traditionally keen on worthwhile audit, and have a track record of successful data collection and analysis. If there is a future in this general plan of large, nationwide collaborative audit organised and conducted jointly by both anaesthetists and surgeons, then it is important that the aims are carefully examined, and the ability of the project to meet these aims is assessed in advance. There are several potential pitfalls to avoid. The first aim of this type of project would be to monitor standards by measurement of outcomes. There has been much anxiety expressed at this proposal [7], and the public and politicians alike will need patient explanation of the statistical inevitability that 50% of any measured performance indicators will inevitably fall below average! However, all good doctors must ensure that their patients derive as much benefit from treatment at their hands as from the hands of another competent doctor, within acceptable limits. Of course, human nature being as it is, there are some doctors who will feel threatened by this approach, but would they feel more or less threatened if an agency other than their own profession were to conduct the monitoring process? No good doctor need fear such assessment. There may be some who are genuinely worried that their results may be poor – which may or may not be down to their own performance. Therefore, it is important that it is clear that results come from a team, which may include the anaesthetist, surgeon, intensive care unit and ward nurses amongst others, or even a whole hospital, including senior management. This is another reason for a collaborative approach to such audit; if an anaesthetist or surgeon alone were to report poor results, then it is possible that suspension from duties would rapidly follow pending investigation. If subsequent investigation were to show that problems exist in, for example, surgical technique, then it is right that those problems be addressed by appropriate means. However, deficiency in operative technique is a rare cause of death after elective aortic aneurysm repair [6, 8], and fault may lie with any number of other areas including, for example, level of ward nursing or availability of interventional radiology. The issue of how to deal with genuinely underperforming doctors remains to be resolved, but the matter is at least under discussion [9]. It is well recognised that those teams conducting high-risk surgery have ‘good runs’ and ‘bad runs’. Can we ensure that a ‘bad run’ by an otherwise competent team is not mistaken for poor performance? The answer to this is clearly ‘yes’ if accurate risk stratification is employed, such as the P-POSSUM method of prediction of mortality [10]. This will allow results to be analysed by the CUSUM (cumulative summation) technique, where actual mortality is plotted cumulatively against the expected mortality [11]. Another fear expressed at this type of outcome audit is that high-risk patients will be denied treatment for fear of making the results from that unit look poor. From the above, it will be readily appreciated that such a fear would be groundless, provided enough data are collected for adequate risk stratification. Risk stratification will become more and more accurate as more data are collected and analysed. The second strand of this work is that having collected all these data primarily for the process of risk stratification and outcome measurement, non-randomised observational studies may then be performed. Such research techniques are frequently considered to be a poor relation of the ‘gold standard’ of randomised controlled trials (RCTs), but this view has been challenged, and the validity and safety of RCTs may have been overstated [12]. The value of observational studies arising from local administrative or audit data collection has recently been convincingly demonstrated [13]. There are many unanswered questions that may not be amenable to RCT, either on ethical grounds or because of the complexity of the problem. To organise an RCT of sufficient power to determine, for instance, the value of various techniques employed to preserve renal function during aortic surgery may require multiple centres, standardisation of technique and numerous patient exclusions in order to achieve uniformity between groups. Thus, the patients studied may no longer represent the population of interest, and to which the results will be applied. Support for non-randomised studies has come from Health Technology Assessments (HTA). A systematic review for the HTA concluded that results from a non-randomised study may be as reliable as those from an RCT provided the non-randomised study is large enough and adjustment for baseline differences are rigorous [14]. A separate publication for the HTA concluded that non-randomised studies are useful where RCTs are not feasible, and also in assessing whether the benefits indicated by RCTs were achieved in wider practice. The comment was also made that data collection needs to be improved by adding more focused data for current clinical interests as a way of answering predetermined questions. The process of data collection can be improved by publication of audits and feedback to those recording the data [15]. Thus, one direction for clinical governance in the future is collaborative audit on a national scale, with collection of data to measure performance accurately and conduct observational research along the way. This will initially require individual effort, but as projects become more numerous and mature, the argument for resources in terms of information technology and personnel employed specifically for the purpose of data collection will become irrefutable. Money, as well as energy, will be required for the conversion of the current ineffective fibrillation that is clinical governance to produce a co-ordinated and productive system that fulfils the needs of the medical profession and the public. Whether the beast that emerges from this resuscitation will bear any resemblance to what was originally planned is uncertain, but at least its output may be effective.
No takes yet. Share an insight, caveat, or question.
P. J. M. Bayly (2001) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: