Layer-Wise Capacity Allocation for Low-Rank Vision-Language-Action Fine-Tuning Under Demonstration Budget Constraints

Low-rank adaptation (LoRA) has become a standard technique for fine-tuning large Vision-Language-Action (VLA) models to specific robotic tasks, yet practitioners typically apply a uniform rank across all transformer layers—a choice that ignores the heterogeneous sensitivity of different layers to taskspecific signals. When the demonstration budget is small (5–25 trajectories), this uniform allocation either under-fits expressive layers or over-fits fragile ones, wasting precious supervision. We propose RANKALLOC, a method that automatically distributes a fixed total rank budget across transformer layers based on per-layer Fisher information estimated from a small held-out set of demonstrations. RANKALLOC formulates rank allocation as a constrained integer program, solved in closed form via a waterfilling analogy, and requires no additional training—only a single forward-backward pass over the held-out set. On the LIBERO benchmark and a physical Franka Emika Panda platform across five manipulation tasks, RANKALLOC improves success rate by 6.8–11.2 percentage points over uniform-rank LoRA and by 3.1–5.4 points over recent importance-based pruning baselines, while using the same total number of trainable parameters. Ablation studies confirm that the Fisher-based allocation consistently outperforms heuristic strategies such as top-heavy or bottomheavy rank schedules.