One-Stage Top-k Learning-to-Defer: Score-Based Surrogates with Theoretical Guarantees

We introduce the first one-stage Top-k Learning-to-Defer framework, which unifies prediction and deferral by learning a shared score-based model that selects the k most cost-effective entities-labels or experts-per input. While existing one-stage L2D methods are limited to deferring to a single expert, our approach jointly optimizes prediction and deferral across multiple entities through a single end-to-end objective. We define a cost-sensitive loss and derive a novel convex surrogate that is independent of the cardinality parameter k, enabling generalization across Top-k regimes without retraining. Our formulation recovers the Top-1 deferral policy of prior score-based methods as a special case, and we prove that our surrogate is both Bayes-consistent and H-consistent under mild assumptions. We further introduce an adaptive variant, Top-k (x), which dynamically selects the number of consulted entities per input to balance predictive accuracy and consultation cost. Experiments on CIFAR-10 and SVHN confirm that our one-stage Top-k method strictly outperforms Top-1 deferral, while Top-k (x) achieves superior accuracy-cost trade-offs by tailoring allocations to input complexity.