Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception

Abstract The ventral visual processing hierarchy of the cortex needs to fulfill at least two key functions: Perceived objects must be mapped to high-level representations invariantly of the precise viewing conditions, and a generative model must be learned that allows, for instance, to fill in occluded information guided by visual experience. Here, we show how a multilayered predictive coding network can learn to recognize objects from the bottom up and to generate specific representations via a top-down pathway through a single learning rule: the local minimization of prediction errors. Trained on sequences of continuously transformed objects, neurons in the highest network area become tuned to object identity invariant of precise position, comparable to inferotemporal neurons in macaques. Drawing on this, the dynamic properties of invariant object representations reproduce experimentally observed hierarchies of timescales from low to high levels of the ventral processing stream. The predicted faster decorrelation of error-neuron activity compared to representation neurons is of relevance for the experimental search for neural correlates of prediction errors. Lastly, the generative capacity of the network is confirmed by reconstructing specific object images, robust to partial occlusion of the inputs. By learning invariance from temporal continuity within a generative model, despite little change in architecture and learning rule compared to static input- reconstructing Hebbian predictive coding networks, simply by shifting the training paradigm to dynamic inputs, the approach generalizes the predictive coding framework to dynamic inputs in a more biologically plausible way than self-supervised networks with non-local error-backpropagation. Author Summary Neurons in the inferotemporal cortex of primates respond to images of complex objects independent of position, rotational angle, or size. While feedforward models of visual perception such as deep neural networks can explain this, they fail to account for the use of top-down information, for example when sensory evidence is scarce. Here, we address the question of how the neuronal networks in the brain learn both bottom-up and top-down processing without labels as they are used in the artificial supervised learning paradigm. Building on previous work that explains vision as a process of iteratively improving predictions, learning in the predictive coding network is driven by the local minimization of prediction errors. When trained on sequences of moving inputs, the network learns both invariant high-level representations comparable to those in the inferotemporal cortex of primates, and a generative model capable of reconstructing whole objects from partially occluded input images in agreement with experimental recordings from early visual areas. Advancing the search for experimental hallmarks of prediction errors, we find that error neurons in the higher areas of the network change their activity on a shorter timescale than representation neurons.