Deaths by horsekick in the Prussian army – and other ‘Never Events’ in large organisations

It was arguably the Prussian military that first recognised the notion of a ‘never event’ when it became concerned (in peacetime) about the number of its officers killed by horsekick in the cavalry 2, 3. These were clearly serious incidents, at risk of lowering morale, potentially reflecting badly on the organisation and possibly a result of poor leadership or systems failure. Yet the Prussians had the foresight to collect hard data. Rather than simply blaming the victims for carelessness, the horses for viciousness, or the generals for incompetent leadership, the mathematician Ladislaus Bortkiewicz, a German of Polish descent, studied the death rates over a 20 year period (1875-1894). In a groundbreaking book entitled the ‘Law of Small Numbers’ (a phrase that still resonates in statistical circles) 3, he showed that the data followed the ‘Poisson distribution’ (Fig. 1), the mathematics of which have been discussed several times previously in the pages of this journal 4-6. Bortkiewicz went on to explain how this is a formal mathematical description of random, rare events. In other words, any real distribution (e.g. Prussian horsekicks) that resembles a Poisson in turn can be readily assumed to have arisen through chance events and not as a result of intent or design (e.g. inherent systems failure in the military). Similarly, if Moppett and Moppett have concluded that NHS Never Events are also Poisson-distributed and so random, then does it follow that they have no ‘cause’ and, in turn, that they cannot be prevented? We discuss below why this does not follow. A common Oxford entrance interview question asks candidates: “What is the opposite of ‘random’?” An unhelpful (and likely unsuccessful) answer is simply to say ‘non-random’, but a more constructive approach might lead to a discussion of ‘determinism’. That is, where all predisposing and causative factors to an event are known, its outcome is then predictable based on certain laws of nature. A candidate might reasonably offer the view that something that is deterministic cannot be truly random and they might even touch upon the notion of ‘free will’. However, the two concepts (randomness and determinism) are not as mutually exclusive as they first appear to be. The motion of one billiard ball (or gas molecule) when struck by another is an entirely deterministic event and no statistics are involved. If one knows exactly the mass, velocity, angle of incidence, friction, etc of the striking ball, the subsequent movement of the struck ball can always be calculated precisely. If we tip a bag of balls onto the billiard table we could, given time and knowledge, calculate the movement and final position of each ball. However, using a different approach based on assumption of random behaviour, we can use statistical modelling to estimate the movement of all the balls at once and thereby predict, for example, the relative probabilities that they will end up being scattered away from the point at which they were tipped, versus remaining motionless or forming uniform patterns. This statistical approach requires no detailed knowledge of the forces applied to any single ball. Correspondingly, for example, in physics the temperature or pressure of a gas is a comment on the average velocity of the molecules within it, and not a description of the velocity of any given molecule. If we heat a gas, we cannot know if the velocity of any given molecule will increase; but we can be sure that the average velocity will rise. This mathematical approach is remarkably accurate (as the Nobel laureate physicist Eugene Wigner termed ‘unreasonably so’) despite its assumptions of randomness, in describing otherwise deterministic physical and biological behaviours 7. In this sense, the concept of ‘random behaviour’ arises because we do not possess all the information about each discrete unit we are observing. Nevertheless, we are able to describe very accurately their behaviour using probabilistic modelling applied to all the units collectively. This statistical/probabilistic approach does not undermine – and is entirely compatible with - the fact that the behaviour of each individual unit could be decided deterministically. As humans, we generally choose our partners very carefully; yet the distribution of our alleles is best modelled as if we were mating randomly 8. In this way, randomness (an assumption at the heart of the probabilistic approach) and determinism (the forces that drive an individual event) can co-exist in harmony. Thus, one description of ‘randomness’ is where the collective behaviour of units in a population can be modelled statistically, based on an assumption that each of two or more possible outcomes is equally likely. The ‘opposite of random’ is a pattern of behaviour which is not compatible (i.e. statistically improbable) with this assumption. A concrete example of this distinction is the recent analysis by Carlisle, who concluded that the data of Fujii were very unlikely to have been derived from random sampling, and were therefore more likely deterministically derived 9-12. So, if a Poisson distribution implies that the data follow a random distribution for rare events, we can now turn to our next question: when should we worry? The answer depends on our perspective. If we are the patient (or the family), then none of the statistics matter: the event has happened to us and naturally we are always concerned. Therefore, the hospital quoted in Moppett and Moppett's paper that classed 11 wrong implants in ophthalmic surgery as one Never Event was wrong to do so. Similarly if we are the medical team, then the overall statistics offer little consolation. Just because Never Events across the country are randomly distributed, it does not mean (as discussed above) our own Event did not have an identifiable or preventable set of predisposing factors. It is imperative that we analyse these (root cause analysis, etc) and learn from them 13, 14. If, however, we are health service planners or regulators, then our perspective may differ. We may be more concerned with identifying outlier hospitals or teams with inherent organisational problems (e.g. identifying a ‘Mid-Staffs’ 15). Where the overall data follow a random (Poisson) distribution, this is notoriously difficult to do using statistics alone. The reasons for this are complex. For example, reviewers’ comments on Moppett and Moppett's paper included discussions around Markov chain analysis, Firth-type bias-reduced logistic regression, relogit package in Zelig and the ‘synthetic minority over-sampling technique’. Readers will be relieved to hear we will not rehearse those arguments. Different analyses can yield opposite results, where hospitals identified as poor performers by one method are statistically no different from a hospital marked as a high performer. Problems of interpretation can arise where sample size is inevitably low (as it is with Never Events) and confidence intervals can be wide 16-18. One additional confounder is that the human mind (even an expert one) is tempted to identify clusters as signifying non-random events. Recently, Sir Roger Boyle, Director of the National Institute for Cardiovascular Outcomes Research (and ‘heart tsar’) sent the Medical Director of the NHS some data that indicated a cluster of paediatric cardiac surgery deaths in Leeds, as a result of which operations there were suspended as ‘unsafe’. A later re-analysis (that included the use of Poisson modelling) indicated there was no reason to suppose this was anything other than random variation, and surgery re-commenced (see: http://www.england.nhs.uk/2013/04/12/reports-chs-leeds/). Counter-intuitively, clusters are therefore often reassuring that random processes have occurred. Because we know that tossing a fair coin results, in the long run, in an equal number of heads and tails, it is often erroneously concluded from this that ‘randomness’ means an ordered or equal distribution of outcomes. Figure 2 might represent some hypothetical final positions of billiard balls spilled onto a table, or the geographical distribution of Never Events. It is Panel B, not Panel A, that should reassure us that random (Poisson) processes are at play. Panel A (a uniform distribution) suggests some planning or collusion. So one trigger for worry is if we see unusual, non-Poisson distributions (eg, uniform or bimodal) as these could indicate underlying problems in the system as a whole. Another cause of worry is the absolute rate of Never Events. The Poisson distribution applies to rare, random events. However, this means that it applies equally well to Prussian officers kicked to death by horses as it does to the much less common event of horses kicked to death by their Prussian officers. The distribution of cases where the patient is accidentally awake during surgery is Poisson distributed across hospitals 19, 20. So, probably, too are the cases where the anaesthetist is accidentally asleep 21. The Poisson does not help us identify an acceptable rate of event; it only tells us that all these events arise randomly. To help a little with this problem, Weaver introduced the notion of the ‘surprise index’ (see Appendix), that mathematically separates the concept of ‘rarity’ from the surprise associated with that event arising 22. Not all rare events are surprising: for example, the winner is always surprised on winning the lottery, but it is not a surprise that someone wins the lottery. On the other hand, it is both rare and a surprise to hear that a Prussian officer has kicked his horse to death, or that an anaesthetist has fallen asleep during surgery. When the prevailing event rate increases, the shape of distribution changes gradually from Poisson to resemble a normal (Gaussian) distribution, so this pattern might be a trigger for concern: ie, a rare event has become a common event (Fig. 3). In summary, the interpretation of Moppett and Moppett's paper depends upon one's perspective (patient, physician or regulator) and upon one's understanding of randomness, determinism and probability. Moppett and Moppett have nicely shown that the distribution of Never Events in the NHS – just like that of deaths by horsekick in the Prussian cavalry over a century ago – follows a quasi-Poisson distribution. This means that the events, when viewed in totality, are rare and random. There is no reason to suspect any fundamental systems failure across the NHS or in any one hospital based on the Never Event data alone. Yet, individual events can have specific causes. From an individual perspective we are reminded that, although we exercise our free will and determinism, we are surrounded by factors that can be regarded as random. The purpose of safe practice, embedded in professional guidelines, is to limit the translation of those factors into adverse outcomes and to mitigate any impacts. Thus, the number of car crashes on any given road may well be Poisson-distributed but this should not stop us from wearing seat belts, sticking to the speed limit, or investigating an accident. Accidental awareness may be described by a Poisson model, but this does not mean we stop checking the anaesthetic machine, monitoring end-tidal agent levels, or applying the now-recommended investigation and care pathways if it arises 23. Similarly, Moppett and Moppett may have shown that the distribution of Never Events is random but this should not stop us from applying the World Health Organisation checklists, or from investigating by root cause analysis when an Event happens. In the light of these statistical considerations, it is clear that Never Events are mis-named. They are really events that will always occur, and do so randomly - but from which learning for individuals and teams should emanate. Re-naming them Poisson-, or ideally Bortkiewicz-Events would better encapsulate these ideas, but it is perhaps rather too hopeful to think that this will catch on. Note that ‘Bortkiewicz’ has several spellings in the literature concerning his work. This is the Polish spelling, but his original work (reference 3) was published in the German version. Other versions exist. No external funding and no competing interests declared. where the summation (Pi) is over all the possible outcomes and Pr is the probability that the given rare event actually occurs. A value of between 0 and 1 corresponds to a likely, unsurprising outcome; the larger the surprise value > 1, the greater the surprise. If only the named person buys the single ticket, n = 1 and the surprise is small. If n is large, the surprise is very large. File S1. More information about Figure 2. [Correction: File S1 was added after online publication on 23 November 2015, to provide more information] Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.