Efficient federated LoRA with heterogeneous ranks for VLMs via dual QR–SVD aggregation

Device heterogeneity in federated fine-tuning of vision-language models (VLMs) leads clients to adopt different LoRA ranks, making conventional homogeneous aggregation difficult to apply. Existing approaches either impose a common rank across clients, limiting resource adaptivity, or reconstruct full-dimensional matrices update with singular value decomposition (SVD), resulting in substantial server-side computational overhead. We propose FedHRQS, a resource-aware federated LoRA framework that supports client-specific rank selection and heterogeneous-rank aggregation through sample-weighted factor concatenation. FedHRQS applies dual thin QR decomposition to the concatenated LoRA factors and performs SVD on the resulting compact core matrix. We show that this design yields the same truncated-SVD solution as directly decomposing the full aggregated update matrix, while substantially reducing server-side complexity by restricting SVD to a compact matrix whose dimension is determined by the total LoRA rank rather than the full model dimension. We further derive a generalization bound characterizing the effects of client-specific rank truncation and effective model dimensionality. Experiments with LLaVA-1.5-7B on LLaVA-ReCap-118K show that FedHRQS improves Google BLEU by up to 40.2% over competitive baselines and achieves 597 × –7586 × speedups in server-side aggregation, which demonstrates that FedHRQS effectively accommodates device heterogeneity while enabling accurate and computationally efficient federated fine-tuning of VLMs.