Vector‐Based SparseGPT: A Vector‐Based Sparsity Method for Large Language Model

Large model sparsity methods improve inference efficiency by removing unimportant weights, but existing methods are often affected by significant accuracy degradation and typically require sparsity‐aware fine‐tuning to restore accuracy. To address this problem, we propose Vector‐Based SparseGPT(V‐SparseGPT), a high‐performance semistructured sparsity framework that eliminates the need for fine‐tuning by integrating the strengths of Optimal Brain Surgeon (OBS) and SparseGPT. Unlike the left‐to‐right iterative pruning strategy in SparseGPT, V‐SparseGPT operates on vector‐level units, enabling simultaneous selection and pruning of multiple weights while updating the remaining parameters, thereby mitigating error accumulation. To further enhance efficiency, we introduce two acceleration strategies: block‐sparsity, which divides weight matrices into smaller submatrices for independent pruning, and layer‐parallel sparsity, which records intermediate activations and performs parallel pruning across layers. Extensive experiments on Qwen2.5 and LLaMA series models demonstrate that V‐SparseGPT consistently outperforms SparseGPT. Under the 2:4 sparse pattern, it achieves up to a 32.91% reduction in perplexity and surpasses the fine‐tuning‐based approach Wanda++ on most language tasks, validating its effectiveness and robustness.