포트레이트토커: 참조 이미지 없이 생성하는 텍스트-음성 기반 3d 말하는 얼굴

Customizable digital avatars that can be specified by text and animated directly from speech are an important building block for virtual agents, educational media, telepresence, and scalable digital human production. Existing talking-head generation methods, however, usually depend on reference images, manual rigging, or subject-specific 3D templates, which limits scalability and personalization. We present PortraitTalker, an end-to-end framework that generates photorealistic 3D talking heads directly from text prompts and speech input. The proposed pipeline combines SDS-based text-to-3D synthesis, a transformer-based speech encoder that predicts FLAME expression and pose parameters, and a differentiable renderer that produces temporally coherent videos. Experiments on the HDTF dataset show that PortraitTalker achieves an LSE-C of 7.230, an LSE-D of 7.712, and an FID of 21.997. In a user study, the proposed method is preferred in terms of lip-sync accuracy (68.13%), motion diversity (76.89%), video sharpness (74.06%), and overall naturalness (74.76%). These results demonstrate that high-quality 3D talking avatars can be generated without reference images or manual rigging, providing a practical path toward scalable avatar creation.