Marco Cattaneo's contribution to the Discussion of “Safe Testing” by Grünwald, de Heide, and Koolen

This is an excellent paper, neatly summarizing previous results about evariables, as well as presenting important advances.The possibility of obtaining conservative p-values and confidence intervals even under optional continuation is a substantial advantage of e-values over alternative measures of statistical evidence.Criticisms of the use of p-values often include the advice that p-values should only be interpreted jointly with an estimate of the effect size of interest.Under optional continuation, effect size estimation may be difficult.With preplanned sequential testing, standard estimators (such as maximum likelihood estimators) can be severely biased, but it may be possible to estimate their bias and improve the effect size estimates.In the optional continuation scenario, even the concept of bias seems problematic, since the sequential analysis is not preplanned and unconditional expectations cannot be easily evaluated.Given an effect size of interest δ and an e-variable E δ0 for each null hypothesis δ = δ 0 , the convex hull of {δ : E δ ≤ 1/α} is a conservative (1 -α) confidence interval for δ.In analogy with likelihood ratios, it seems quite natural to consider arg min δ E δ as an estimate of the effect size δ.Have the properties of such "minimum e-value" estimators been studied, in particular under optional continuation?The end of Section 6 discusses the sample size requirements in situations in which stopping is possible after each data point.It is shown that in order to achieve the desired power, hypothesis test based on e-variables may need on average less data than standard tests with fixed sample size.It would be interesting to include also standard sequential tests in the comparison, since they would also be possible in such situations and may also need on average less data than tests with fixed sample size.