Handling results below the level of detection

In this issue, we publish a paper by Flynn (2010), which adds to the extensive literature on how to handle results which lie below the level of detection (non-detects), including the paper we published by Krishnamoorthy et al., (2009) a few months ago. This led us to ask Helsel (2010) to contribute a commentary on the topic. Helsel concludes with four ‘things that I think we should be able to agree on’. Perhaps the most controversial of these is that the substitution method is so flawed compared to other methods that journals should ‘reject papers that use it. The one exception might be when estimating the mean for data with one censoring threshold …’. The substitution method involves replacing every result below the level of detection (D) with an estimate, or rather a guess, for what it might be. At least four systems have been used, and all produce bias and imprecision in calculated parameters of the results distribution. Sometimes zero or D has been substituted for each missing value but this leads to particularly erroneous results, and it is hard to see that these approaches can be defended. More commonly, D/2 or is used. Helsel's four recommendations led to a debate in the Editorial Board as to whether we should indeed reject papers that substitute D/2 or D/2⁠. The conclusion was that we should not necessarily do so. The key principle as with all measurements and data treatments is that the conclusions must be justified by the evidence, or, to put it another way, approximations in the data treatment must not be so gross as to undermine the validity of the conclusions. So on substitution, researchers should be familiar enough with the problems to be able to justify their approximations and not just substitute because it is easy. We hope that Helsel's commentary will help with this. Guides to the effects of approximations have been provided by Singh et al., (2006) and by Hewett and Ganser (2007). For example Hewett and Ganser's Table 2 shows that if the percent of non-detects is between 1 and 50 and the number of observations between 20 and 100, for a log-normal distribution with geometric standard deviation between 1.2 and 4, then substitution of D/2 or D/2 means that the average result shows fairly modest bias in the 95th percentile or the mean, and the imprecisions of these estimates are only slightly worse than the optimum method of data treatment. This is the ‘one exception’ allowed by Helsel (2010) in the above quote. Hewett and Ganser also deal with more complicated cases, for example in combining results from several labs with different limits of detection, and in such cases, the errors introduced by simple substitution become more important. The real problem is that researchers need accessible methods that they can use reasonably easily. This is the attraction of the substitution methods, although it can never justify using an approach which is inadequate for the conclusions. As Helsel notes, Flynn enables his method within Excel, which should encourage its use. Also, Finkelstein and Verma (2001) gave a method of implementing maximum likelihood estimation with a spreadsheet. If other workers can point to simple ways of implementing other methods, we will be interested to hear about them.