Despite advancements in artificial intelligence, object recognition models still lag behind in emulating visual information processing in human brains. Recent studies have highlighted the potential of using neural data to mimic brain processing; however, these often rely on invasive neural recordings from non-human subjects, leaving a critical gap in understanding human visual perception. Addressing this gap, we present, ‘Re(presentational)Al(ignment)net’, a vision model aligned with human brain activity based on non-invasive EEG, demonstrating a significantly higher similarity to human brain representations. Our innovative image-to-brain multi-layer encoding framework advances human neural alignment by optimizing multiple model layers and enabling the model to efficiently learn and mimic the human brain’s visual representational patterns across object categories and different modalities. Our findings demonstrate that ReAlnets exhibit stronger alignment with human brain representations than traditional computer vision models, achieving an average similarity improvement of approximately 3% and a maximum relative improvement ratio reaching up to 40%. This alignment framework takes an important step toward bridging the gap between artificial and human vision and achieving more brain-like artificial intelligence systems. EEG-aligned fine-tuning makes artificial vision models more brain-like, enhancing model-human representational similarity across EEG, fMRI, and behavior.
Lu et al. (Fri,) studied this question.