A 2D Gabor-wavelet baseline model out-performs a 3D surface model in scene-responsive cortex

Abstract Understanding 3D representations of spatial information, particularly in naturalistic scenes, remains a significant challenge in vision science. This is largely because of conceptual difficulties in disentangling higher-level 3D information from co-occurring features and cues (e.g., the 3D shape of a scene image is necessarily defined by “low-level” spatial frequency and orientation information). Recent work has employed newer models and analysis techniques that attempt to mitigate these in-principle difficulties. For example, one such study reported 3D-surface features were uniquely present in areas OPA, PPA, and MPA/RSC (areas typically referred to as ‘scene-selective’), above and beyond a Gabor-wavelet baseline (“2D”) model. Here, we tested whether these findings generalized to a new stimulus set that, on average, dissociated static Gabor-wavelet baseline (“2D”) features from 3D scene-surface features. Surprisingly, we found evidence that a Gabor-wavelet baseline model better fit voxel responses in areas OPA, PPA and MPA/RSC compared to a model with 3D-surface information. This raises the question of whether previous findings of greater 3D information could have been due to a baseline condition that didn’t model some potentially critical low-level features (e.g., motion). Our findings also emphasize that much of the information in “scene-selective” regions—potentially even information about 3D surfaces—may be in the form of spatial frequency and orientation information often considered 2D or low-level, highlighting the fundamental conceptual challenges associated with disentangling the contributions of low-level vs. high-level features in visual cortex. Author summary To gain a more complete picture of human visual processing, it is critical to understand the precise format of representations of naturalistic visual scenes. Recent work has approached this challenge by quantifying how much of our brain activity might be due to hypothesized characteristics of the stimuli being viewed. Here, we followed up on work finding that activity in scene-responsive regions of the brain is well predicted by information about the 3D configurations of major surfaces in viewed scenes, like walls and floors. In contrast to previous work, we found that our baseline condition—commonly thought of as “low-level” visual information—accounted for these regions’ responses better than both of the 3D surface models we tested. First, we argue that this difference in results could be due to differences in the baseline conditions used across studies. However, our findings do not necessarily argue against the importance of 3D surface information. Instead, they highlight the possibility that baseline models can perform well by virtue of covariation between low-level features like orientation and spatial frequency with higher-level properties like depth information. This motivates future work and/or new analysis frameworks to better characterize the interplay between hypothesized model features and the specific techniques used to quantify their relationship to neural representations.