Answers are grounded in this paper and cite their sources.
This paper audits one repository-specific compute-normalization rule for LoRA rank comparison. The rule allocates optimizer updates inversely with rank, but it is an operational proxy rather than a measurement or matching of floating-point operations, accelerator time, or energy. The recovered evidence comprises 30 head-only baseline artifacts and 8 LoRA artifacts on SST-2. We add 56 controlled LoRA runs that hold the effective scaling ratio at α /r=2 , separate seed from rank, retain and count the jointly trained classifier head, and record GPU runtime. On SST-2 with , the inverse 682/42/10/1-step schedule yields mean accuracies of 0.7909, 0.5700, 0.5046, and 0.4969 for ranks 2, 8, 16, and 64 across three shared seeds. With 42 updates for every rank, five-seed means instead rise with rank: 0.5422, 0.5961, 0.7211, and 0.7475. The same equal-step pattern appears with on SST-2 (0.6307, 0.7454, 0.7943, 0.8024) and with on AG News (0.5645, 0.7093, 0.7852, 0.8242), each across three shared seeds. A warmed timing benchmark gives 100.21–101.18 ms per update across ranks, showing that the extreme update schedule does not match measured runtime: its total time is dominated by the number of updates. The contribution is diagnostic rather than a broad rank recommendation. This normalization rule can make larger ranks look worse through severe undertraining, and the observed ordering is not stable when scaling, seed, and update opportunity are controlled.
No takes yet. Share an insight, caveat, or question.
Tang et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: