Answers are grounded in this paper and cite their sources.
Training Recurrent Spiking Neural Networks (RSNNs) with binary spikes for tasks of extended time scales presents a challenge due to the amplified vanishing gradient problem during back propagation through time. This paper introduces three crucial elements that significantly enhance the memory and capabilities of RSNNs, with a strong emphasis on compatibility with hardware and neuromorphic systems. Firstly, we incorporate neuron-level synaptic delays, which not only allow the gradient to skip time steps but also reduce the overall neuron population’s firing rate. Subsequently, we apply a biologically inspired branching factor regularization rule to stabilize the network’s dynamics and make training easier by incorporating a time-local error in the loss function. Lastly, we modify a commonly used surrogate gradient function by increasing its support to facilitate learning over longer timescales when using binary spikes. By integrating these three innovative elements, we not only resolve several complex benchmarks but also achieve state-of-the-art results on the spiking permuted sequential MNIST task (psMNIST), showcasing the practicality and relevance of our approach for digital and analog neuromorphic systems.
No takes yet. Share an insight, caveat, or question.
Balafrej et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: