Determinism and Scale in LLM Attribution: Lessons from the NTLLM Challenge

The NTLLM challenge ("Name That LLM") asks competitors to attribute short text passages to one of 18 frontier LLMs, scored by balanced accuracy on a leaderboard whose public split covers roughly 40% of the hidden test set. We report three measured findings from a solo, compute-rationed campaign (one shared T4 GPU tier, 30 GPU-hours per week, competition deadline January 14, 2027). First, determinism is scarce at the platform level and expensive to manufacture elsewhere — but where it matters, it is achievable. A byte-identical (MD5-matched) fork of the public 0.668 frontier kernel scored 0.661 on resubmission, with content-level prediction diffs diagnosed between runs: platform rerun variance of ±0.009 exists even when code, data, and weights are bit-for-bit identical. Controlled two-answer-per-prompt simulations on two independent held-out splits then showed that every calibration and draw-sampling knob in the frontier lineage (Sinkhorn rebalancing, target mixing, the typography/sibling/pairwise post-chain) has zero-to-negative, non-replicable effect. We additionally document that the lineage's entire post-processing chain has never executed in any scoring rerun — a latent `NameError` after `submission.parquet` is written — so every published score in the 0.651→0.668 lineage is the raw blend argmax. Second, scale of training data, not decoding tricks, is the lever. Top-team disclosure (a 0.675 finisher) attributes the edge to privately generated LLM data. Public pools max out at ~211k rows; we built and published a deduplicated 419k-row mega-pool covering all 18 classes, and train a Qwen3-14B LoRA member on it using a capped-trunk (first 27 layers) configuration that fits a single T4. Third, our reusable contribution: a bitwise-exact checkpoint/resume protocol that converts a fixed weekly GPU ration into an arbitrarily long training schedule — "train right to your quota." The protocol checkpoints every 500 steps or 25 minutes with atomic swap, serializes optimizer, scheduler, scaler, heads, and all four RNG streams, force-saves and exits cleanly at 11.2 hours (ahead of Kaggle's 12-hour session wall), and resumes across sessions by self-sourcing the kernel's own committed output. Correctness was proven bitwise-exact on a zero-GPU-cost CPU probe before any quota was spent, and the production run demonstrated the full lifecycle: clean failsafe exit at step 2366/2890, checkpoint committed to output. Current honest standing: leaderboard 0.661, rank 1533/5253, climb in progress; the mega-pool LoRA member (v34) is trained with this protocol and its leaderboard verdict is pending.