Amoeba: An Efficient and Flexible FPGA-Based Accelerator for Arbitrary-Kernel CNNs

Inspired by the key operation of vision transformers (ViTs), convolutional neural networks (CNNs) have widely adopted arbitrary-kernel convolutions to achieve high performance in diverse vision-based tasks. However, existing hardware efforts primarily focus on implementing CNN models that consist of a stack of small kernels, which poses challenges in supporting large-kernel convolutions. To address this limitation, we propose Amoeba, a flexible field-programmable gate array (FPGA) -based inference accelerator designed for efficiently supporting CNNs with arbitrary kernel sizes. Specifically, we present an optimized dataflow approach in collaboration with the Z-flow method and kernel-segmentation (Kseg) scheme, which enables flexible support for arbitrary-kernel convolutions without sacrificing efficiency. Additionally, we incorporate vertical-fused (VF) and horizontal-fused (HF) methods into the layer execution schedule to optimize the computation and data transfer process. To further enhance the CNN deployment performance, we employ the loop tiling scheme search (LTSS) method, guided by a fine-grained performance model, during the early design phase. The proposed Amoeba accelerator is evaluated on Intel Arria 10 SoC FPGA. The experimental results demonstrate excellent performance on prevalent and emerging CNNs, achieving a throughput of up to 286. 2 GOPs. Notably, Amoeba achieves 4. 36 better DSP efficiency compared to prior works on the same network, highlighting its superior utilization of hardware resources for CNN inference tasks.