Pose-robust representation for face verification in unconstrained videos

Face recognition in unconstrained videos has been actively studied with the increasing popularity of surveillance and personal cameras. Compared with the traditional task of face recognition in images, recognizing faces in unconstrained videos is much more challenging due to the large variations in poses, expressions and lighting conditions. To handle varying poses in the videos, in this paper, we propose a two-level representation approach for face verification in unconstrained videos. Specifically, we first recover the full-pose face representation for each video which contains all the pose categories ranging from the most left to the most right profile faces. The missing poses are synthesized using keyframes of known ones. Then, we further propose a cross-pose video pair representation for face verification task, which consists of the similarity scores of all frame-level pairs across two videos. Extensive experiments on the benchmark YouTube Faces (YTF) dataset clearly demonstrate the effectiveness of our proposed method.