Relative faces: Encoding of family resemblance relative to gender means in face space

Neurophysiological (W. A. Freiwald, D. Y. Tsao, D. A. Leopold, I. V. Bondar, G. Rhodes R. Robbins, E. McKone, & M. Edwards, 2007) evidence suggests that faces are encoded as differences from a mean or prototypical face, consistent with the conceptual framework of a mean-centered face space (T. Valentine, 1991). However, it remains unclear how we encode facial similarity across classes such as gender, age, or race. We synthesized Caucasian male and female cross-gender "siblings" and "anti-siblings" by projecting vectors representing deviations of faces from one gender mean into another gender. Subjects perceived male and female pairings with similar vector deviations from their gender means as more similar, and those with opposite vector deviations as less similar, than randomly selected cross-gender pairings. Agreement in relative direction in a space describing how facial images differ from a mean can therefore provide a basis for perceived facial similarity. We further demonstrate that relative coding for male and female faces is based on the activation of a shared neural population by the transfer of an identity aftereffect between a face and its cross-gender sibling. These results imply whereas structural similarity may be reflected in the Euclidean distance between points in face space configural similarity may be coded by direction in face space.