Hierarchical learning creates invariant schema within plastic neural networks

Cognitive neural circuits must balance the plasticity needed for continual learning with the stability needed to preserve an underlying reasoning framework, called a schema. How circuit learning rules form and protect such schemata from being continually overwritten during learning remains unknown. On a visual boundary detection task, we show that a hierarchical learning algorithm creates an invariant schema circuit whose weights remain fixed following sparse initial training, with additional data serving to refine the upstream representation. In contrast, the end-to-end backpropagation algorithm used to train nearly all current artificial neural networks comprehensively changes its weights throughout training. We show that the hierarchical schema makes an independent mechanistic hypothesis about circuit computation that is consistent with experimental ordering observed in single-neuron measurements in humans. These results suggest that hierarchical learning is sufficient to encode biologically consistent persistent cognitive models within otherwise malleable neural networks.