A Two Stage Generative Model for Controllable Human Head Motion Video Generation

This is the authors' abstract. We don't add key points for this paper.

Recent advances in diffusion models have greatly enhanced image and video generation quality, finding applications in virtual reality, gaming, and digital media. A key challenge in this field is generating realistic and expressive animated portraits with controlled motion, which existing methods often lack. To address this, I propose a framework that combines a trajectoryguided head pose prediction module, the Diff Transformer, with AniPortrait for generating high-quality animations driven by audio, static images, and user-defined trajectories. The Diff Transformer maps trajectory inputs to head pose sequences using Differential Attention, while AniPortrait ensures temporal consistency and visual quality. Evaluations show that our framework significantly improves pose prediction accuracy and maintains high video quality, as confirmed by user studies that highlight strong satisfaction with motion coherence and video quality. This approach enhances controllability and expressiveness, making it ideal for applications in personalized character animation, interactive media, and dynamic storytelling.