Face Swapping under Large Pose Variations: A 3D Model Based Approach

Traditional face swapping technologies require the faces of source images and target images have similar pose and appearance (usually frontal). This limits its applications. This paper presents a method for face swapping based on personalized 3D head models. This framework builds a personalized 3D head model from a frontal face and can be rendered at any pose to match the characters in the image we want to swap. The 3D head model is constructed by a user uploaded frontal view face image. This construction process goes through face alignment and feature point matching. The final personalized 3D head is built by deforming a standard 3D head model using radial basis function to match the specific person. To make the synthesized face seamlessly blended into the image, color transfer and multi-resolution spline technique are applied. We use the proposed technique to create personalized storybook where the characters are replaced with a user's face and promising results are obtained. The system can be used in face de-identification as well.