We appreciate the initiative of Addiction to reprint the Executive Summary of the committee’s final report and to solicit remarks on the report from a set of distinguished commentators. The Executive Summary is informative but can do no more than profile the main elements of our analysis, conclusions, and recommendations. We hope that readers of Addiction will be sufficiently interested in the report to want to see it in its entirety. This is easily accomplished, as the report is available online from National Academy Press (http://www.nap.edu/catalog/ 10021.html). The mission of the committee ended formally with the release of the final report, so there can be no ‘official’ committee response to the comments solicited by Addiction. However, we three members of the committee have concluded jointly that two of the comments warrant responses. These follow. We are also grateful to the other commentators (Bammer 2002; Kandel 2002; Kleber 2002; Klingemann 2002; Maynard 2002; Reuter 2002; Weatherburn 2002). Two important conclusions of the NRC Report are that data on the quantities of illegal drugs consumed by users in the United states are non-existent and that the available data on drug prices are deeply flawed. Accordingly, the Report recommends that work be started to acquire data on consumption and better data on prices. Reliable data on consumption and prices are central to virtually any effort to assess the effectiveness of drug control policies. Yet many drug policy analysts and Federal agencies with drug policy and enforcement responsibilities are unwilling to think seriously about how such data might be acquired. Reuter, for example, argues that: There is ‘a good deal of information in STRIDE’ because STRIDE prices are strongly associated with hospital emergency room admissions and self-reported use of drugs by high-school seniors. Moreover, these associations are consistent with theory. The problems of acquiring better data are difficult to overcome because of a need to rely on informants to buy drugs and because ordinary buyers do not know the quantity and purity of the material they purchase. There must be an effort to learn what information is contained in STRIDE and how best to use it. Reuter’s third argument is an important reason for acquiring better data, although this may not be the interpretation that he intended. Better data make it possible to estimate and, thereby, correct errors made by past data-acquisition methods. Highly respected Federal statistical agencies such as the Bureau of Labor Statistics work hard to improve their data acquisition procedures and to make any required adjustments in past data. There is no reason why the same cannot be done with data on drug prices. The first argument is also incorrect. Two data series can be highly correlated but very different. A forthcoming article (Horowitz 2001) gives an example of two price series for cocaine base that are derived from STRIDE. The two series are highly correlated, but they do not always move in the same direction from one year to the next, and their levels differ by up to a factor of two. No appeal to theory can alter this fact. A data series that does not indicate reliably whether prices increase or decrease from one year to the next cannot be relied upon for use in, for instance, estimation of price-elasticities of demand or the effectiveness of enforcement actions, regardless of how highly correlated it is with variables such as hospital admissions. It is true that acquisition of better data is a difficult task, but the outlook is not as bleak as Reuter suggests. It is not necessary to rely on informants to make purchases. The DEA and local police, for example, rely mainly on undercover agents because informants cannot testify in court. More importantly, however, law-enforcement agencies have missions and cultures that are incompatible with acquisition of price or other data for statistical and analytical purposes. One reason for this is that it is dangerous for law-enforcement officers to work in drug-dealing neighborhoods and to buy drugs because the officers are dangerous to the dealers. The committee’s report makes several suggestions for ways that data on prices and consumption might be acquired without the need for purchases by law-enforcement officers or informants. It also makes suggestions for ways of dealing with buyers’ lack of quantitative knowledge of the price and purity of the drugs they purchase. Enforcement activities are the main component of drug control policy in the United States. These activities are expensive and disruptive. They may increase the violence associated with drug markets. Yet the most fundamental data needed to evaluate their effectiveness are not available. It is time to start thinking seriously about how this lack of data can be remedied. Dr Kleber’s comments misconstrue the committee’s arguments about treatment evaluation. It is now commonplace for advocates in the treatment community to cite specific benefit–cost ratios asserting that treatment more than pays for itself in social benefits. It is also commonplace for advocates in the treatment community to cite specific estimates implying that treatment is significantly more cost– effective than enforcement, interdiction or source-country controls as a means of reducing drug use. Both claims may well be true, but both are premised on uncontrolled pre-post estimates of the reduction in drug use following drug treatment. Those estimates are vulnerable to two plausible and well-known statistical threats to validity: selection biases and regression to the mean. If the treatment community wishes to make such claims, they need to provide credible evidence that the claims are in fact valid. Credible benefit–cost and cost–effectiveness estimates will be of great value in informing policy debates about how to allocate drug-control funds. They are also needed to address a crucial policy question: what would happen if we were to scale up our existing treatment system significantly to provide help to those who are not currently receiving the services they need? Obviously, randomized trials with no-treatment controls pose difficult logistical, ethical and even legal questions, but these obstacles do not seem insurmountable. Still, assume for the sake of argument that such trials were impossible. Would this therefore eliminate the relevance of the inferential threats we identified? Of course not; but it would require the treatment community to significantly revise their strong assertions about the specific benefit–cost and cost–effectiveness advantages of treatment, or else provide stronger evidentiary foundations for those assertions. At any rate, the committee did not assert that every study needs no-treatment controls; indeed, a mere handful of such studies, in different settings and with different clients, would go a long way to helping us clarify the contribution of selection processes and regression artifacts in treatment evaluation research, and such studies hardly need drive out other forms of treatment research. Indeed, the committee strongly recommended several other approaches, including more use of ‘treatment A versus treatment B’ randomized trials, the use of more sophisticated statistical analyses of non-randomized observational studies (e.g. waiting list studies, quasi-experimental designs) and the use of meta-analyses to synthesize existing data on treatment selection and attrition processes and the situational and client factors that might moderate those processes.
No takes yet. Share an insight, caveat, or question.
Horowitz et al. (2002) studied this question.
Synapse has enriched 2 closely related papers on similar clinical questions. Consider them for comparative context: