Exploring changes in treatment effects across design stages in adaptive trials

The recently published Committee for Medicinal Products for Human Use reflection paper on flexible designs highlights a controversial issue regarding the interpretation of adaptive trials. The guideline suggests that a test for heterogeneity should be preplanned and if treatment effect estimates differ significantly between design stages then data collected before and after the interim analysis might not be combined in a formal analysis. In this paper we investigate error rates for such a procedure in the presence of calendar-time effects. Furthermore, we present an alternative testing strategy based on change point methods. In a simulation study we demonstrate that our procedure performs well in comparison to that suggested by the guideline.