Evidence for Fall Prevention in Hospitals

To the Editor: Accidental falls are the commonest reported patient safety incident in hospitals; predominantly affect older people; and are associated with a variety of underlying pathologies, physical and psychological harms, anxiety for families and staff, systems costs, and risk management concerns.1 A focus on the prevention of such falls and fall injuries, as provided by the excellent systematic review and meta-analysis by Coussement et al.2 is welcome, but the article also illustrates the problems of interpreting systematic reviews as a basis for guiding clinical practice and services or prompting future research. A recent survival analysis of 100 systematic reviews showed that reviews with at least one significantly different outcome measure had superseded approximately one-quarter of the initial reviews within 2 years.3 Does this mean that institutions who have started fall prevention programs should now withdraw them because of the “lack of evidence”? In short, it all depends—on which studies are included; which census date is chosen; how the studies are (and whether they should be) aggregated; which patients are selected for intervention; for multifaceted interventions, what is in the “black box” of that intervention (and whether the balance of components is the same in any two trials); and which outcomes are chosen and how they are ascertained or open to biases. When studies with clinically heterogeneous populations and interventions are pooled for meta-analysis, whether a null result automatically negates strong evidence from individual studies specific to those settings must also be considered.4 Caution must also be exercised regarding the “external validity” of successful interventions if applied to hospitals or units in different health systems or cultures or with different staffing or case-mix. Finally, because conventional randomized, controlled trials (RCTs) are difficult to perform in this setting (hospitals) and this population (frail older people with multiple comorbidities or cognitive impairment, often short stay), whether “lower” levels of evidence such as “before and after” studies should be discounted even when they are of high quality.5 All of this is potentially confusing for staff and institutions wishing to implement best practice. A systematic review and meta-analyses4 on fall and injury prevention in institutions (including hospitals) was reported in 2007 and concluded that there was evidence for a modest effect (∼18%) on fall rate ratios for multifaceted interventions in hospitals, with no overall effect on fractures or relative risk of falling. The main rationale provided by Coussement et al. for their JAGS review2 published only 12 months later was that the 2007 study had the following “limitations.” First, it stated that this review “failed to list criteria for inclusion.” (In fact, it listed them quite explicitly—simply intervention studies of all designs in hospital settings, for which fall rate ratios, relative risk of falling, and fracture rate ratios could be calculated.) Second, it stated that that it “failed to consider the quality of studies.” (Actually, the Downs and Black quality score was explicitly employed and used to order the studies in forest plots.) Third, it stated that it failed to make clear “what constituted the samples.” (The characteristics of study populations are clearly set out in the “Web extra” tables.) A more valid potential criticism would be of the deliberate decision to include and pool studies that were not RCTs. To exclude such studies might have led to the rejection of useful sources of evidence for practice in a field in which “criterion standard” RCTs are hard to perform. I have no such criticisms of the JAGS study, which also incorporated two studies reported after the census date of the original study,6, 7 but readers should be cautious in taking home to their own institutions the message that fall prevention programs in hospital do not work after all, because: There have been two major studies (of high quality within their class) with large numbers of patients using “before and after” design and continuous quality improvement programs8, 9 with large and statistically significant effects on falls rate ratio, relative risk of falling, and rate ratio for injurious falls in one study9 and on falls rate ratio alone in the other.8 Trials of this design may be technically considered inferior sources of evidence to even small and low-quality RCTs, because they are potentially liable to confounding due to recording bias or changes in case-mix or staffing and tend to overestimate effect when compared with RCTs.10 But it is not credible that the effect sizes reported in such large study populations were all simply artefactual. And the authors took a variety of steps to adjust for confounding factors as explanations for effect. Moreover, these studies using whole-systems long-term approaches (both over 3 years) reflect the kind of real-life interventions based on continuous quality improvement that hospitals could replicate. And their sample sizes dwarf the pooled sample from meta-analysis. This leads to an anomaly; because these studies were not RCTs, Coussement et al. did not include them, even though they did include small and poor-quality RCTs. This illustrates the hazards of overreliance on Cochrane-style methodology in providing meaningful evidence for practitioners. Through no fault of the authors, since their census date, there has been a further high-quality RCT of strength-balance training as a single intervention in hospital care (in addition to usual care) with similar effect size on falls rate ratio.11 Despite adjustments for statistical heterogeneity, the pooling of clinically heterogeneous interventions in clinically heterogeneous populations means that potential messages from successful interventions in individual studies are diluted by pooling for meta-analysis. After all, a fair-quality, complex, multifaceted intervention which may or may not be targeted at subgroups of already often very specific populations does not become ineffective merely because it is pooled with trials of different quality, using different interventions in different populations. In short, although this is undoubtedly a good piece of work, and the earlier review had flaws of its own, it is difficult to produce a straightforward and credible conclusion from meta-analysis in such a complex area. Multifaceted fall interventions in frail, acutely ill older people with multiple comorbidities are not the same as drug trials for single conditions with tight exclusion criteria. It depends on what you choose to include and how you choose to analyze the data. Conflict of Interest: The editor in chief has reviewed the conflict of interest checklist provided by the author and has determined that the author has no financial or any other kind of personal conflicts with this letter. Author Contributions: This was a research letter in response to a systematic review in JAGS January 2008 and based on my own previous extensive research in this field. The letter was entirely my own work. Sponsor's Role: No sponsorship involved.