FWHT-RVV: A RISC-V vector processor with FWHT instruction extension for edge LLM inference

The deployment of large language models (LLMs) on edge devices is constrained by computational and memory overhead. While rotation-based low-precision quantization addresses some limitations, matrix-multiplication-based rotation operations with O(N2) complexity introduces new performance bottlenecks. Replacing these operations with the Fast Walsh–Hadamard Transform (FWHT) reduces computational complexity to O(Nlog2(N)), but this method remains inefficient on scalar processors. To address this, we propose FWHT-RVV, a RISC-V vector processor with FWHT instruction extension for edge LLM inference. The proposed approach employs a blockwise hybrid FWHT algorithm to minimize data permutation and reduce memory access overhead. Furthermore, we introduce specialized vector instructions to reduce data movement costs. Experimental results demonstrate a 2.55x average speedup and a 2.52x improvement in energy efficiency compared to the baseline vectorized implementation, with only a 0.80% increase in hardware area.