Horizontally tiled network of cortico-basal ganglia-like modules performs reinforcement learning

The neocortex and basal ganglia nuclei are connected along regions that share the same topography and are orderly arranged. Inspired by the anatomical characteristics of the cerebrum, we developed a network in which the modules of the neocortex-basal ganglia unit were arranged in a horizontally tiled manner. By applying this network to reinforcement learning tasks, we demonstrated that reinforcement learning can be achieved through horizontal signals passing between modules. Each module not only performs its calculation but also provides signals to adjacent modules. This lateral transmission takes advantage of the differences in the projection ranges of the three basal ganglia pathways, the direct, indirect, and hyperdirect pathways, which have been examined in physiological studies. We found that these differences enabled temporal-difference-error computations. This study proposes a computational proof-of-concept to explore whether horizontal tiling and differential projection ranges, inspired by anatomical and physiological feature of basal ganglia, can support reinforcement learning.