The basic concept of “attention heads” in Transformers matches with partial features of “islets of inter-LINKed spine heads,” a model of nervous system functions

In a model of the nervous system, spatial relations between specific signals arriving from input neurons are registered in “islets of inter-LINKed spine heads (IILSHs)”, whose spines belong to different output neurons. In response to a prompt (cue stimulus), operation of IILSHs generates both first-person inner sensations and motor outputs such as speech and behavior. Present work shows that operation of attention heads within the hidden layer located in between input and output neuronal layers of Transformer in large language models is equivalent to a linear algebraic treatment of one segment of operation of IILSHs that generates motor outputs.