A Critique of Bayesian Methods for Retrospective Exposure Assessment

Received 8 January 2002; There have been many informal discussions among occupational exposure assessors about application of Bayesian methods for retrospective exposure assessment and that is why we were pleased to see the appearance in print of a thoughtful and methodical article by Ramachandran (2001), a follow-up to an earlier publication (Ramachandran and Vincent, 1999). In this letter we wish to address several issues raised in that paper (Ramachandran, 2001) that require, in our opinion, a more critical assessment than that provided by the author. We hope that in doing so, we will further stimulate the discussion surrounding this important issue. The author describes and illustrates a Bayesian approach to quantitative exposure assessment in which priors derived from expert judgements and physical models are updated by available exposure measurements. There are several reasons why the proposed method can be expected to fail. They stem from several errors in judgement by the author, including (i) underestimation of errors and biases in expert evaluations, (ii) overestimation of accuracy and utility of exposure models in real workplaces, (iii) application of arguments developed for individual-based exposure assessment to group-based exposure assessment and (iv) undervaluing of the usefulness of exposure measurements available to most exposure assessors. First, expert input is prone to at least as much random error, and probably more systematic bias, than exposure measurements. The claim that professional hygienists are well calibrated to assess exposure (p. 656) is generally not correct. As we have demonstrated earlier, occupational hygienists should be calibrated in order to end up with similar results (Kromhout et al., 1987; Post et al., 1991). The author’s own results support this notion: in figure 5 a systematic difference in estimated average exposures can be seen of up to a factor of 10. The reported correlation on p. 666 with the average measured exposure is not relevant in this case, but the estimate of bias (not reported) is. By combining expert-based prior distributions (instead of treating them as independent assessments), the author also missed an opportunity to accurately reflect uncertainty in expert assessment. The combined error bars of expert-based distributions would cover a similar range of values as shown for the measured exposure levels. Lastly, the problem with the comparison carried out in figure 5 lies in the fact that the distribution of individual exposure measurements is being compared with distributions of average exposures estimated by experts and physical models. In other words, experts did not take into account within- and between-worker variability, while it is reflected in the distribution of personal exposure measurements. Secondly, deterministic and statistical models rarely explain more than 50% of total variability in exposure (Burstyn and Teschke, 1999). Thus, unlike the author’s claim, we are far from having accurate generic exposure models that fit all workplaces. Even if accurate deterministic and statistical models were available for workplaces, the information on determinants of exposure is usually harder to obtain than measurements (Burstyn et al., 1999, 2000; Burstyn and Kromhout, 2000; Wameling et al., 2000). In any case, the uncertainty in this information cannot be assumed to be smaller than the uncertainty in measured exposure levels. The author virtually forced the uncertainty in determinants of exposure to be low, resulting in little influence of exposure measurements on the posterior distributions. The author further undermined the usefulness of the available measurements when he artificially increased their variability two-fold (p. 658). Furthermore, some determinants of exposure that appear to be strong predictors of exposure levels in laboratory studies can fail to have similar predictive power in actual workplaces. One such example is the application temperature of asphalt, which was a strong predictor of bitumen fume emissions in a laboratory rig (Brandt and De Groot, 1999) but was not identified as an important predictor of exposure to bitumen fumes during actual road paving (Burstyn et al., 2000). Thirdly the author proposes what is essentially a group level exposure assessment, where all members of an occupational title in a given time period and location are assigned the same exposure distribution. For this approach it is not necessary to accurately quantify variability, because only valid measures of central tendency (average, median) are needed to produce an unbiased slope of an exposure–response relationship (Berkson, 1950). Thus, the proposed shrinkage estimators are of limited theoretical value for quantitative exposure–response modelling in the application considered by the author. As we have already stated, bias stemming from biased or uncertain input data in physical models may well be a bigger problem. Consequently, a reduction in the variance of exposure estimates per se is not a useful criterion to judge the performance of a group-based exposure assessment methodology. Fourthly, the author seems to undervalue exposure measurements. Even though the author admits that ‘if the measurements had a relatively small amount of uncertainty…expert judgement may be quiet unnecessary’, he never discusses how one would judge that the uncertainty is relatively (to what?) small and why one would need expert judgement at all in such a case. It is ironic that when listing (in the penultimate sentence) pieces of information required for quantitative exposure assessment the author does not mention exposure measurements! Remarkably, in the example of nickel smelter operations used by the author, he had 828 measurements available over a 19 yr period, a data set of a size that most exposure assessors only dream of, and yet these data alone was pronounced to be inadequate for quantitative exposure assessment. Certainly a more comprehensive analysis of the data was warranted before making such a harsh judgement as ‘retrospective exposure reconstruction based solely on such historical measurements leads to estimates with such large error bars as to not be useful for developing quantitative dose–response relationships for epidemiology’. We would like to remind the readers that even in the absence of historical exposure measurements, current measurements have been shown to be very useful in retrospective exposure assessment (Dodgson et al., 1987; Kromhout et al., 1995; Hornung et al., 1996). Nonetheless, we recognize the utility of Bayesian methods in exposure assessment, but propose that their development and evaluation should proceed in a more rigorous manner. Therefore, we suggest evaluating the proposed approach in epidemiological analyses and in simulation studies with large exposure databases, emphasizing a reduction in bias, rather than variance. There is also a need to formally develop an alternative paradigm in which exposure measurements (or predictions of statistical models derived from them) are used as priors, updated by expert assessment and/or application of deterministic exposure models. This is similar to the approach adopted in our study of asphalt workers, where exposure estimates and trends estimates in statistical models where ‘adjusted’ by experts (Burstyn, 2001). Our expert-based adjustments to statistical models did not follow a rigorous mathematical treatment, but can (and should) be formalized in a Bayesian framework. Such an approach would be more conducive to promotion of evidence-based occupational hygiene and exposure assessment. In conclusion, we believe that the author overstates the usefulness of his Bayesian approach to exposure assessment. In his own assessment (p. 665–6) there are only a few special (and rare) cases where the proposed approach would not be ‘a waste of time’. Thus, we believe that evidence to date demonstrates that, when faced with a lack of or extremely poor quality exposure measurement data, exposure assessors have to face the fact that only qualitative, or at best semi-quantitative, exposure reconstruction is possible. Bayesian approaches should not revive expert judgement as methods of choice in exposure assessment, because they cannot turn expert assessment into a method for quantitative exposure assessment. In an era when more and more quantitative exposure measurements become available for the purposes of both occupational hygiene and epidemiology, we should utilize them, instead of falling back on expert judgement.