Abstract Model merging offers a training-free solution to the prohibitive storage costs of the one-model-per-task paradigm. However, current merging techniques present a difficult trade-off. Creating a single model is storage-efficient but hurts performance, while adapter-based methods perform well but introduce a new storage burden. This work introduces CF-STAR, a framework that resolves this dilemma by creating highly compressible adapters. Our approach redefines the adapter itself. Instead of representing the deviation from a pre-trained model, our adapter is defined by the deviation from the multi-task average. We call this new representation the centralized task vector (CTV). This CTV represents a purer form of task-specific knowledge within the merging context, making it fundamentally more compressible. CF-STAR exploits this with a novel low-rank plus sparse decomposition tailored to this representation, capturing both global structure and critical details. Furthermore, the entire pipeline is designed to be synergistic with low-bit quantization, further enabling extreme compression. On diverse benchmarks spanning image classification, NLP, and dense prediction, CF-STAR sets a new state of the art on the accuracy-storage Pareto frontier, achieving up to 40 40 × adapter compression over strong baselines while maintaining competitive performance.
Wu et al. (Sat,) studied this question.