Transformers are increasingly being deployed at the edge to support agentic AI, real-time processing, and privacy-preserving applications. However, their substantial computational and memory requirements often exceed the energy and latency constraints of edge devices. Through detailed characterization, we observe that the majority of computations and parameters in transformer models are concentrated in the feed-forward Multi-Layer Perceptron (MLP) blocks, identifying them as a key target for optimization. In this work, we introduce ET, a novel transformer architecture tailored for energy-efficient edge inference. By integrating shared Look Up Table (LUT) based feed-forward blocks with traditional self-attention modules, ET significantly reduces computational load, memory footprint, and energy usage, aligning closely with the operational and resource constraints of edge devices. We present the design, implementation, and evaluation of ET, and demonstrate that it delivers comparable accuracy to traditional transformers while achieving 2.1 × improvement over a baseline ASIC accelerator. It also achieves a 1.12 × efficiency on GPU acceleration, and about 1.34 × improvement over a prior BinaryBERT FPGA accelerator. We further report 1.2 × energy efficiency over recent multiplication-free models, such as Bitnet, showing that ET offers a compelling and practical solution for deploying transformers effectively at the edge, whether on FPGAs, ASICs or edge GPUs.
Nag et al. (Fri,) studied this question.