Answers are grounded in this paper and cite their sources.
A split-conformal threshold calibrated with one numerical implementation of a classifier is often reused with another: a lower-precision weight copy, a different matrix-multiply mode or a different inference backend. We audit what that reuse does to prediction sets on 30 frozen multilayer perceptrons (Digits, MNIST, Fashion-MNIST; two architectures; five seeds) executed under 15 configurations, each derived independently from the original FP32 weights. A control first shows that an earlier version of our pipeline mutated weights in place: re-applying that mutation reproduces the historical CUDA FP32 outputs exactly for 30/30 models, so those historical GPU discrepancies were a pipeline artefact. In the corrected experiment, numerical changes still alter prediction sets while leaving top-1 predictions unchanged: with the probability score at α = 0.1, CPU BF16 changed 120 test-set prediction sets across the 30 model evaluations, 119 of them without an argmax change. Such changes are rare (largest group-mean rate 0.107%), and the largest absolute five-seed mean coverage difference is 0.102 percentage points. Finally, ranking unlabeled inputs by their source-side distance to the calibration threshold concentrates these rare disagreements under a deployment-query budget: at 100 queries it beat the exact random expectation in 113 of 420 model/runtime cases and tied in the remaining 307 zero-discovery cases, at the cost of scoring the whole source pool. The primary score and query budget were selected after earlier test results had been inspected. These exploratory results describe this benchmark.
No takes yet. Share an insight, caveat, or question.
S. M. Abdulla Al Mamun (2026) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: