Bitmap-Based Sparse Matrix-Vector Multiplication with Tensor Cores

Sparse matrix-vector multiplication (SpMV) plays a crucial role in various scientific and engineering tasks. Thus, extensive research efforts are devoted to enhancing its performance. In this work, we investigate the utilization of the tensor cores — hardware originally designed for dense matrix multiplications — for SpMV. By reverse engineering the architecture of tensor cores, we gain important insights into their internal register layout and develop a technique for direct access to these registers. Building on these findings, we propose Spaden, a method for accelerating SpMV using tensor cores. Spaden comprises two main components: (1) a bitmap-based format that achieves compression for the sparse matrix while preserving its rectangular shape, and (2) a pairing kernel, facilitating efficient execution of SpMV on tensor cores by enabling precise register-level control. Performance evaluation are conducted on Nvidia V100 and L40 GPUs. Compared with state-of-the-art approaches, Spaden demonstrates substantial speed advantage and memory efficiency, achieving, for example, a 1.63 × speedup and 2.83 × memory saving over cuSPARSE CSR on Nvidia L40.