Seeking consensus by formal methods: a health warning

There are compelling arguments for the creation and use of guidelines to steer clinical practice: the range of interventions and therapies on offer is large and ever increasing; more effective treatments should be preferred over the less effective; costs are often high and resources are limited so funds should be focused where they will do the most good for the most patients. Some doctors are antipathetic to guidelines—which they decry as ‘cookbook medicine’—but there is now general acceptance that we cannot afford a free-for-all in clinical practice. The authors believe that there should be equitable access to health care. Expenditure on relatively ineffective treatments, and treatment at high cost of those who can pay to the detriment of those who cannot, run counter to this principle. In order to adjudicate on what should and should not be done, we need transparent processes. There are several systems advocated for the grading of evidence,1 with randomized controlled trials gaining an A grade in an A, B, C grading. There are many aspects of care which rely on lower levels of evidence but when the cause and effect relationship between treatment and outcome is clear, the beneficial effect is large, and cost is supportable, then a strong recommendation (grade 1 of 2) can still be made. Hip replacement, cataract surgery and valve replacement for aortic stenosis are all in this league. Trials of effectiveness are not justified at this stage, and the ‘sky diving without a parachute’ analogy2 can be invoked. They earn 1C recommendation: that is, a strong recommendation with poor quality evidence.1 There are many instances where evidence of benefit is much less clear or the effect size is marginal. Where high level evidence runs out, increasing use is made of expert panels. In this paper we illustrate pitfalls in the expert panel process as used for ratings of appropriateness of interventions for the very common clinical problem of malignant pleural effusion,3 and point to areas for future development.