High Accuracy Power Optimized FPGA Implementation for GELU Function: A Chebyshev Approach

The Gaussian Error Linear Unit (GELU) is widely adopted in modern deep learning models, especially Transformer architectures, due to its smooth gating behavior and favorable convergence properties. However, efficient hardware realization is difficult since GELU embeds both a cubic term and the tanh(·) function, which are costly for FPGA and ASIC implementations. Existing approaches, such as lookup tables (LUTs), piecewise linear (PWL) approximations, and CORDIC iterations, simplify computation but suffer from storage overhead, reduced accuracy, or iterative latency, limiting their use in high-performance accelerators. This paper proposes a Chebyshev polynomial approximation of tanh(x) within the GELU function, evaluated using the Clenshaw recurrence. The method ensures minimax error distribution while avoiding LUT storage and iterative operations, making it well-suited for fixed-point FPGA datapaths. Validation on an Xilinx Artix-7 FPGA shows a mean absolute error of 10−4 and maximum error below 10−3 compared with MATLAB’s (V.Ra2025) GELU, delivering roughly 25–45× lower error than PWL-based designs and about an order of magnitude lower than polynomial-LUT hybrid designs. Resource and power results further demonstrate the efficiency of the proposed design, establishing Chebyshev-based GELU as an accurate and scalable solution for FPGA and ASIC accelerators.