AI - ISP Accelerator with RISC-VISA Extension for Image Signal Processing

This paper focuses on the hardware acceleration of an AI-ISP (image signal processing) neural network, namely CSANet, which computes images from raw data to RGB format. To design a flexible, high-performance, and low-power AI-ISP accelerator on FPGA, we apply the RISC-VISA extension with the CFU-Playground framework specified for tiny-ML. The proposed RISC-V custom instructions support normal convolution, transposed convolution, depth-wise convolution, and partial element-wise operations, which can accelerate the computation hotspots of the CSANet. Applying with the FPGA implementation, on the Digilent Nexys Video development board (Artix-7 XC7A200T), the proposed CSANet accelerator achieves a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">79.7×</tex> speedup and demonstrates a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">27.8×</tex> improvement in energy efficiency on average compared to RISC-V CPU-only.