MIRACLE: Mutual-Information Regularized Alignment and Compression Control for Robust Vision-Language-Action Policy Learning

MIRACLE technical report / preprint. Finetuning vision-language-action (VLA) models for control can improve in-distribution action prediction while degrading visual-semantic structure inherited from pretrained backbones. This work studies the phenomenon through an information-theoretic lens, asking when action-only finetuning discards transfer-relevant information and how that compression can be controlled without freezing the backbone. VLA finetuning is formulated as a constrained optimization problem in which the policy remains predictive of actions while preserving selected information in intermediate and deep representations. Under assumptions on representation smoothness, task sufficiency, and support mismatch between pretraining and robot finetuning data, action-only training can induce biased compression of pretrained features that are weakly required in-distribution but useful under shift. The authors propose MIRACLE (Mutual-Information Regularized Alignment and Compression Control for Robust VLA Finetuning), combining layerwise preservation via a consistent InfoNCE-based surrogate, mid-layer anchor alignment using centered kernel alignment (CKA), and insulated action adaptation through lightweight adapters and selective gradient scaling. A local constrained-training analysis is provided with a closed-form initialization for layerwise dual variables under a Gaussian-linear approximation, refined online via projected dual ascent. Controlled simulations support three consistent patterns: deeper layers are more vulnerable to action-driven drift, moderate preservation improves shifted robustness, and overly strong preservation impairs adaptation. The primary contribution is a theory-method framework for studying representation degradation in VLA finetuning, together with a concrete algorithm and fully specified simulation evidence. Existing OSF archival DOI: 10.17605/OSF.IO/NPRHY; Existing OSF archival page: https://osf.io/nprhy/. Files include the technical report PDF and the LaTeX source tarball when available.