Optimization and FPGA implementation of Echo State Networks to predict the chaotic Lorenz system

This work presents an optimized Field-Programmable Gate Array (FPGA) implementation of an Echo State Network (ESN), a type of Recurrent Neural Network (RNN), to predict the chaotic signal of the Lorenz system. The focus is on executing AI models efficiently on resource-limited devices like those in the Internet of Things (IoT). In contrast to previous state-of-the-art works that used the tanh activation function (which required complex hardware approximations), this proposal employs the simpler ReLU function. This choice eliminates the need for approximations and, when combined with an optimized network architecture, enables a drastic reduction in resource usage. Specifically, the optimization achieved three key reductions: a) The reservoir size, decreased from 50 to 3; b) The connectivity matrix, transitioned from a dense matrix of 2500 values to a diagonal matrix with only 3 non-zero values; and c) Precision, reducing the operational bit-width from 32 to 23 bits. As a result, the optimized ESN is significantly more efficient on FPGA hardware, successfully reducing utilized resources while simultaneously achieving improved performance compared to prior state-of-the-art implementations. • We apply a Network Search Architecture with Echo State Networks (ESN) to predict the chaotic signal of Lorenz system. • A very small network with a size of 3 neurons in the reservoir is ob- tained. • An implementation of the obtained ESN in FPGA is presented.