Abstract Selection and ranking of multiple groups are fundamental problems in many scientific and practical applications. This paper develops a Nonparametric Predictive Inference (NPI) framework for selection and ranking based on a single future observation from each group. The focus is on deriving NPI lower and upper probabilities for events concerning the identification and ordering of the best groups. Selection within subsets of independent groups is first studied, where ‘better’ is defined as the event that all future observations from the selected groups exceed those from the remaining groups. Exact NPI lower and upper probabilities are derived for selecting the two best groups, and approximate probabilities are obtained for selecting the three best groups. The methodology is subsequently extended to ranking groups organised into buckets, each containing one or more independent groups. Data-driven allocation of groups to buckets and the choice of the number of buckets are investigated, where the optimal configuration is defined as the one that optimises the relevant NPI lower and upper probabilities. An NPI-Bootstrap approach is also employed to approximate the probability of the event of interest. Simulated data and examples from the literature illustrate the proposed methods. The proposed framework provides a flexible and assumption-lean approach to selection and ranking, offering informative probability bounds that explicitly quantify predictive uncertainty.
Alshihry et al. (Mon,) studied this question.