A Reduced Spiking Neural Network Architecture for Energy Efficient Context-Dependent Reinforcement Learning Tasks

Neuromorphic circuits and systems involving spiking neural networks (SNN) have resulted in disruptive advances in performance/joule for relevant applications. A novel reinforcement learning (RL) digital hardware architecture is presented in this work that achieves energy consumption improvements through three fundamental techniques: The first two techniques comprise reduction in the complexity of the arithmetic unit for the optimization of recurring synapse and neuron cores in the network array, which is inspired by "crude" nature of the building blocks in the biological neurons that are tolerant to inaccuracy and noise. As the third technique, the RL SNN middle (hippocampus) layer is equipped with a simple scratchpad to facilitate temporal hysteresis in synaptic plasticity during the RL processes of long-term potentiation/depression (LTP / LTD). This feature is inspired by the non-temperamental behavior of biological synapses. The intelligent allocation significantly reduces learning time in a given task. Implementation on Intel Cyclone IV FPGA demonstrated significant advantages in cost, power dissipation and execution time, resulting in more than two orders of magnitude benefit in energy consumption for a context-dependent learning task on a 16-node 3-layer RL network presented in the literature.