Abstract Cognitive neuroscience bridges insights into human brain mechanisms with artificial intelligence, where brain-inspired architectures have driven unprecedented success in artificial neural networks. However, endowing AI models with the dynamic functional patterns of biological brains remains a longstanding challenge. To advance computational models that better emulate the brain’s information processing, this study systematically investigates semantic representations in the human visual cortex and constructs a biologically plausible framework for categorizing embedded semantic concepts in visual stimuli. We designed a controlled cognitive experiment to analyze visual semantic processing, collecting fMRI data from 15 participants. A spatiotemporal graph network was employed to capture dynamic features of semantic brain regions, enabling the construction of functional networks for concept classification and prediction. Leveraging self-supervised learning, our decoding framework reconstructs visual stimuli and compares them with predicted categorical outputs to derive semantically coherent representations. Experimental results demonstrate the model’s superiority in decoding fMRI data, outperforming existing methods in both accuracy and semantic consistency. This unified framework integrates visual and semantic processing, offering biologically interpretable insights into brain-inspired semantic cognition.
Zheng et al. (Mon,) studied this question.