Decoding Sound and Imagery Content in Early Visual Cortex

Human early visual cortex was traditionally thought to process simple visual features such as orientation, contrast, and spatial frequency via feedforward input from the lateral geniculate nucleus (e.g., 1Spillmann L. Phenomenology and neurophysiological correlations: two approaches to perception research.Vision Res. 2009; 49: 1507-1521Crossref PubMed Scopus (48) Google Scholar). However, the role of nonretinal influence on early visual cortex is so far insufficiently investigated despite much evidence that feedback connections greatly outnumber feedforward connections 2Salin P.A. Bullier J. Corticocortical connections in the visual system: structure and function.Physiol. Rev. 1995; 75: 107-154PubMed Google Scholar, 3Markov N.T. Vezoli J. Chameau P. Falchier A. Quilodran R. Huissoud C. Lamy C. Misery P. Giroud P. Ullman S. et al.Anatomy of hierarchy: feedforward and feedback pathways in macaque visual cortex.J. Comp. Neurol. 2014; 522: 225-259Crossref PubMed Scopus (364) Google Scholar, 4Self M.W. van Kerkoerle T. Supèr H. Roelfsema P.R. Distinct roles of the cortical layers of area V1 in figure-ground segregation.Curr. Biol. 2013; 23: 2121-2129Abstract Full Text Full Text PDF PubMed Scopus (112) Google Scholar, 5Singer W. Cortical dynamics revisited.Trends Cogn. Sci. 2013; 17: 616-626Abstract Full Text Full Text PDF PubMed Scopus (124) Google Scholar. Here, we explored in five fMRI experiments how information originating from audition and imagery affects the brain activity patterns in early visual cortex in the absence of any feedforward visual stimulation. We show that category-specific information from both complex natural sounds and imagery can be read out from early visual cortex activity in blindfolded participants. The coding of nonretinal information in the activity patterns of early visual cortex is common across actual auditory perception and imagery and may be mediated by higher-level multisensory areas. Furthermore, this coding is robust to mild manipulations of attention and working memory but affected by orthogonal, cognitively demanding visuospatial processing. Crucially, the information fed down to early visual cortex is category specific and generalizes to sound exemplars of the same category, providing evidence for abstract information feedback rather than precise pictorial feedback. Our results suggest that early visual cortex receives nonretinal input from other brain areas when it is generated by auditory perception and/or imagery, and this input carries common abstract information. Our findings are compatible with feedback of predictive information to the earliest visual input level (e.g., 6Mumford D. On the computational architecture of the neocortex. II. The role of cortico-cortical loops.Biol. Cybern. 1992; 66: 241-251Crossref PubMed Scopus (753) Google Scholar), in line with predictive coding models 7Friston K. The free-energy principle: a unified brain theory?.Nat. Rev. Neurosci. 2010; 11: 127-138Crossref PubMed Scopus (3398) Google Scholar, 8Clark A. Whatever next? Predictive brains, situated agents, and the future of cognitive science.Behav. Brain Sci. 2013; 36: 181-204Crossref PubMed Scopus (2522) Google Scholar, 9Bar M. The proactive brain: using analogies and associations to generate predictions.Trends Cogn. Sci. 2007; 11: 280-289Abstract Full Text Full Text PDF PubMed Scopus (800) Google Scholar, 10Bastos A.M. Usrey W.M. Adams R.A. Mangun G.R. Fries P. Friston K.J. Canonical microcircuits for predictive coding.Neuron. 2012; 76: 695-711Abstract Full Text Full Text PDF PubMed Scopus (1178) Google Scholar.