Abstract The computational substrate of biological systems exhibits remarkable abilities to learn complex skills quickly and efficiently. Inspired by this, we implement model-based reinforcement learning (MBRL) using spiking neural networks (SNNs) directly on mixed-signal neuromorphic hardware. This approach combines energy-efficient electronic circuits with high sample efficiency through alternating online (``awake'') and offline (``dreaming'') learning phases.Our model features two networks: an agent network that learns from real and simulated experiences and a world model network that generates simulated experiences.We validate this by training the system to play Atari Pong. First, we establish a baseline using only real experiences. Then, by ``dreaming'', the required real experiences decrease significantly.The network dynamics runs in real-time on the analog neuromorphic circuits, with only the readout layers implemented and trained on a computer-in-the-loop. We present results that demonstrate the robustness and potential of energy-efficient mixed-signal neuromorphic processors for real-world applications.
Blakowski et al. (Tue,) studied this question.