Audio-Driven Talking Face Video Generation with Emotion

Vivid talking face generation has potential applications in virtual reality. Existing methods can generate talking faces that are synchronized with the audio, but typically ignore the accurate expression of emotions. In this paper, we propose an advanced two-step framework to synthesize talking face videos with vivid emotional appearances. The first step is designed to generate emotional fine-grained landmarks, including the normalized landmarks, gaze, and head pose. In the second step, we map the facial landmarks to latent key points, which are then fed into the pre-trained model to generate high-quality face images. Extensive experiments demonstrate the effectiveness of our method.