A Full-Digital Heterogeneous Computing-in-Memory With Dual-Bit Serialized Weight Stream and Ping-Pong Computing for Efficient Transformer Acceleration

Vision Transformers (ViTs) have demonstrated remarkable performance across various computer vision tasks. However, the self-attention mechanism in transformers incurs high processing loads and costly off-chip data movement in conventional von Neumann architectures. This paper proposes a full-digital heterogeneous Computing-in-Memory (CIM)in-situhybridizing differential RRAM and SRAM to enable efficient on-chip weight storage and flexible row-column dual-directional access for efficient transformer acceleration. A Ping-Pong computing dataflow is introduced to maximize the throughput density by allocating static and dynamic computations across CIM macros. A low-toggle weight tuning algorithm is proposed to minimize the dynamic power under the dual-bit-serial CIM scheme by introducing only small weight rounding distance. Besides, a patch-channel skip mechanism is used to reduce the redundant computing for dynamic matrix with high sparsity. Experimental results on different transformer models show that the proposed DBS-CIM architecture reduces layer-wise computing energy by up to 59.9% compared to conventional digital CIM. Compared to prior homogeneous CIM designs, it achieves2.0×to3.3×higher end-to-end energy efficiency, and outperforms existing heterogeneous CIM designs by2.13×in computational density.