The State of Modeling Face Processing in Humans with Deep Learning

Recent studies show that ‘deep learning’ facial recognition can potentially model face processing in humans. In this review, we summarise insights from this research and compare existing deep learning architectures to psychological models of the face processing system. Psychological models consist of two components: (i) a core component that extracts ‘face codes’ from images for separate processing of identity and dynamic information; (ii) an extended component that links these codes to broader person information, multimodal cues, and social context. Initial work has examined face codes produced by deep convolutional neural networks (DCNN) that are engineered for identity recognition only. Given this singular engineering goal, the observation that DCNN face codes include a multitude of other attributes – for example, illumination, pose, expression and social judgments – challenges psychological models. However, psychological models highlight the limited extent to which DCNNs capture human processing. Deep learning is not currently capable of processing dynamic face information, learning individual familiar faces, nor does it address how face codes in the core component are linked to the extended system. Closer alignment of psychological and engineering approaches is necessary for understanding how face information is encoded and transmitted in the brain.