From efficacy to effectiveness: evaluating psychedelic randomised controlled trials for trustworthy evidence-based policy and practice.

The recent review of a new drug application for MDMA-assisted therapy for post-traumatic stress disorder by the United States’ Food and Drug Administration (FDA) highlighted epistemological and methodological challenges for evidence assessments. Similar challenges will also be faced in reviews of other compounds in early- and late-stage development, like psilocybin for depression. The regulatory demand for two successful phase 3 randomised controlled trials (RCTs) seems problematic, given a current lack of agreement on what constitutes “success”, particularly when psychoactive drug administration is concomitant with (psycho)therapy. These complex arrangements challenge the internal validity of estimated average treatment effect through comparison with conventional control conditions. This paper reviews the assumptions behind RCTs’ current “gold-standard” status in the hierarchy of evidence-based medicine (EBM). Recapitulating known epistemic limits of randomisation and blinding, it emphasises the urgent need to avoid the extrapolation fallacy. The resulting argument is that the degree of trustworthiness that efficacy — reported in RCTs — will reliably predict effectiveness — in target populations outside RCTs — depends on what type of psychedelic treatments will be regulated. If “stand-alone” drugs for large scale prescription and consumption, trustworthiness should be graded low. On the other hand, for regulation of drug-assisted (psycho)therapies, the degree of trustworthiness can be considered high. The reason being that these two treatment approaches are based on different causal claims with distinct external validities. Therefore, careful assessment of support factors in each is recommended to prevent detrimental consequences, from potential rejection of effective therapies up to medical reversal of eventually approved drugs.