REDOMA: Bayesian random‐effects dose‐optimization meta‐analysis using spike‐and‐slab priors

The rise of cutting‐edge precision cancer treatments has led to a growing significance of the optimal biological dose (OBD) in modern oncology trials. These trials now prioritize the consideration of both toxicity and efficacy simultaneously when determining the most desirable dosage for treatment. Traditional approaches in early‐phase oncology trials have conventionally relied on the assumption of a monotone relationship between treatment efficacy and dosage. However, this assumption may not hold valid for novel oncology therapies. In reality, the dose‐efficacy curve of such treatments may reach a plateau at a specific dose, posing challenges for conventional methods in accurately identifying the OBD. Furthermore, achieving reliable identification of the OBD is typically not possible based on a single small‐sample trial. With data from multiple phase I and phase I/II trials, we propose a novel Bayesian random‐effects dose‐optimization meta‐analysis (REDOMA) approach to identify the OBD by synthesizing toxicity and efficacy data from each trial. The REDOMA method can address trials with heterogeneous characteristics. We adopt a curve‐free approach based on a Gamma process prior to model the average dose‐toxicity relationship. In addition, we utilize a Bayesian model selection framework that uses the spike‐and‐slab prior as an automatic variable selection technique to eliminate monotonic constraints on the dose‐efficacy curve. The good performance of the REDOMA method is confirmed by extensive simulation studies.