Adaptation optimizes sensory encoding of future stimuli

Abstract Sensory neurons continually adapt their response characteristics according to recent sensory input. However, it is unclear how such a reactive process shaped by sensory history can benefit the organism going forward. Here, we test the hypothesis that adaptation indeed acts proactively in the sense that it optimally adjusts sensory encoding for the future, i.e. for the next expected sensory input. We first quantified adaptation induced changes in sensory encoding by psychophysically measuring discrimination thresholds for visual orientation under different adaptation conditions. Using an information theoretic analysis, we found that adaptation consistently reallocates coding resources such that encoding accuracy peaks at the adaptor orientation while total coding capacity remains constant. We then asked whether this characteristic change in encoding accuracy is predicted by the temporal statistics of natural visual input. By analyzing the retinal input of freely behaving human subjects in natural environments, we found that the distribution of local visual orientations in the retinal input stream at any moment in time is also peaked at the mean orientation computed over a short input history leading up to that moment. We further tested our hypothesis with a recurrent neural network trained to predict the next frame of natural scene videos (PredNet). We simulated our human adaptation experiment with PredNet while analyzing its internal sensory representation. We found that the network exhibited the same change in encoding accuracy as observed in human subjects, and as predicted by the natural input statistics. Taken together, our results suggest that adaptation induced changes in encoding accuracy are an attempt of the visual systems to be best possibly prepared for future sensory input.