Is color in the human visual cortex chromatic or content-predictable? A confound-controlled decoding study of the Natural Scenes Dataset

In natural scenes, color and content co-vary, so color decoded from brain activity may be inferred from content rather than being chromatically derived. Using the Natural Scenes Dataset (7T fMRI, eight participants), eleven basic color terms were decoded from three voxel-count-matched regions: early visual cortex (V1–V3), V4, and higher visual cortex. Higher visual cortex led on the chromatic terms (ridge; R² = 0.081, against 0.052 in early cortex) and early cortex on the achromatic ones (0.057, against 0.040) — a double dissociation, confirmed against a separate luminance target (interaction ΔΔR² = +0.023, p = 0.008, 8 of 8), consistent with the graded preference from controlled stimuli (Goddard & Mullen, 2020). This pattern held across linear and nonlinear decoders, tracked retinotopic eccentricity rather than the dominant object, and ran counter to signal quality. Residualizing against object presence (80 COCO categories) reduced decoding more in higher than early cortex (ΔΔR² = +0.021, 8 of 8). The loss was specifically chromatic: the chromatic lead of the higher visual cortex disappeared, while the early cortex kept its achromatic lead (8 of 8). The same held for a perceptual color-naming target, and after removing 91 scene categories. As a control analysis, the same residualization applied to the luminance target left a content-unpredicted component in the early cortex alone (8 of 8). Chromatic color decodable from higher visual cortex is therefore largely content-predictable, whereas early cortex’s achromatic signal is not — bearing on predictive-coding accounts and models reconstructing color from brain activity.