Benchmarking proton RBE models

Abstract Objective. To biologically optimise proton therapy, models which can accurately predict variations in proton relative biological effectiveness (RBE) are essential. Current phenomenological models show large disagreements in RBE predictions, due to different model assumptions and differences in the data to which they were fit. In this work, thirteen RBE models were benchmarked against a comprehensive proton RBE dataset to evaluate predictions when all models are fit using the same data and fitting techniques, and to assess the statistical robustness of the models. Approach. Model performance was initially evaluated by fitting to the full dataset, and then a cross-validation approach was applied to assess model generalisability and robustness. The impact of weighting the fit and the choice of biological endpoint (either single or multiple survival levels) was also evaluated. Main results. Fitting the models to a common dataset reduced differences between their predictions, however significant disagreements remained due to different underlying assumptions. All models performed poorly under cross-validation in the weighted fits, suggesting that some uncertainties on the experimental data were significantly underestimated, resulting in over-fitting and poor performance on unseen data. The simplest model, which depends linearly on the LET but has no tissue or dose dependence, performed best for a single survival level. However, when fitting to multiple survival levels simultaneously, more complex models with tissue dependence performed better. All models had significant residual uncertainty in their predictions compared to experimental data. Significance. This analysis highlights that poor quality of error estimation on the dose response parameters introduces substantial uncertainty in model fitting. The significant residual error present in all approaches illustrates the challenges inherent in fitting to large, heterogeneous datasets and the importance of robust statistical validation of RBE models.