Abstract Background and Aims A Study Within A Trial (SWAT) is a research study embedded within a larger trial which aims to investigate different strategies for a particular trial process, such as trial recruitment. It is imperative such studies, which are often underpowered, employ efficient and informative analytical methods. The use of Bayesian methods and interpretation of SWAT results in this context requires exploration. Bayesian methods provide direct probability statements about intervention effects and readily enable ACceptability Curve Estimation using Probability above Threshold (ACCEPT) analyses, which consider the probability of the tested intervention being effective for different threshold values. Additionally, they provide an opportunity to incorporate the results of previous similar studies within the analysis using informative priors. This proof-of-concept study re-analysed two previous SWATs using Bayesian methods and ACCEPT analyses. Methods A SWAT conducted by Du et al. in 2009 and a subsequent SWAT by Mattock et al. in 2020 compared a video intervention against standard patient information on trial recruitment. For each SWAT, a primary Bayesian analysis was performed using a logistic model with non-informative priors. Sensitivity analysis explored informative priors informed by meta-analysis of previous similar studies; this included an analysis of the Mattock et al. SWAT incorporating the result of the earlier Du et al. SWAT. ACCEPT curves were constructed. Results were compared with frequentist analyses. Results For the Du et al. SWAT, the primary Bayesian analysis gave an OR for recruitment for the video relative to standard information of 2.12, 95% CrI: 0.38–4.65 and a posterior probability of the video being effective (OR > 1) of 0.86. When taking into account results of previous SWATs by using an informative prior there remained a moderately high probability of video benefit (0.82). For the latter Mattock et al. SWAT, the primary Bayesian analysis gave an OR for recruitment for the video relative to standard information of 0.26, 95% CrI: 0.07–0.51 and the posterior probability of the video being effective was 0.0005, indicating very little chance of effectiveness; ACCEPT plots facilitated interpretation by showing the probability that the video was better than standard information for OR > 0.8 was very small (0.0032). When taking into account the results of previous SWATs using an informative prior, including Du et al., the probability of the video being effective was still very small (0.12). Conclusions Bayesian methods and ACCEPT analyses offer solutions to challenges experienced in the analysis and interpretation of SWATs, which are often underpowered. Greater use of these analytical approaches within SWATs will lead to a more accessible, improved evidence base on how to effectively conduct trials.
Cro et al. (Fri,) studied this question.