Randomized trial examines compute-optimal training deadlines in machine learning, indicating new insights for model scaling.
Paper supported by machine-verified Lean 4. Compute-optimal scaling laws (Kaplan et al.; Hoffmann et al., "Chinchilla") prescribe a model-size/data-size split that minimizes loss for a fixed FLOP budget, implicitly assuming the training run may take arbitrarily long wall-clock time to realize that budget. Real training is instead bound by a hard deadline: a competitive release date, a funding window, a fixed rental period on a GPU fleet. We show that once training is required to finish within a wall-clock time tau, McCandlish et al.'s gradient-noise-scale theory of the steps/examples tradeoff becomes formally an endoreversible heat engine: batch size plays the role of a throttle, the number of optimizer steps plays the role of cycle time, and there is an absolute, un-crossable wall-clock floor tau_rm floor=S_min,t_1,N below which no finite batch size — however large — can complete training. The resulting data efficiency eta(tau)equiv D_min/D_rm used(tau) takes the exact Carnot functional form eta(tau)=1-tau_rm floor/tau, with the deadline tau playing the role of a hot reservoir and tau_rm floor the role of a cold one. Pricing wall-clock time against extra data tokens and minimizing total cost yields a closed-form golden-rule deadline tau^ast=tau_rm floor+sqrtc_D D_mintau_rm floor/c_tau — a genuine Curzon–Ahlborn-flavored square-root correction, derived rather than assumed, from the same generic near-a-critical-floor optimization that produces efficiency-at-maximum-power in physical heat engines. The theorem directly explains, in direction and mechanism, a 2026 empirical finding that wall-clock-constrained optimal model size grows differently than Chinchilla's FLOP-based prediction. All results are numerically verified (21/21); for an illustrative 18-billion-parameter model the golden-rule deadline sits at approx 30 days against an approx 18-day absolute floor, at a striking approx62% Deadline Tax in extra compute. We cast the construction as a new Intelligence Bound self-application, the Stoker, close with four testable predictions, and discuss implications for AI training infrastructure, conservation-model retraining under field-season deadlines, and the canon's own thermodynamic-economics program.
No takes yet. Share an insight, caveat, or question.
Hart et al. (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: