What question did this study set out to answer?

This work aims to create a framework for generating co-speech gestures in virtual avatars using audio input.

June 1, 2023

Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generation

View Full Paper

Ask AI

Bookmark

Authors

Randomized trial demonstrates improved gesture quality in audio-driven avatars, indicating advancements in human-machine interaction.

This work aims to create a framework for generating co-speech gestures in virtual avatars using audio input.
Developed Diffusion Co-Speech Gesture framework to capture audio-gesture associations.
Implemented Diffusion Audio-Gesture Transformer for improved modality attention and temporal dependency modeling.
Introduced Diffusion Gesture Stabilizer with an annealed noise sampling strategy for temporal consistency.
DiffGesture achieved state-of-the-art performance in generating coherent gestures with improved mode coverage.
Showed stronger audio correlations in generated gestures across different audio inputs.

Zhu et al. (2023) studied this question.

View Full Paper

Ask AI

Bookmark