Enhancing Intangible Cultural Heritage Dance Formation Choreography in VR via Large Language Models

The recent popularity of consumer-grade virtual reality devices has enabled users to experience immersive Intangible Cultural Heritage (ICH) dances in virtual environments, where ICH is defined by UNESCO as traditions or living expressions inherited from our ancestors. As in real-world choreography, the dancers in a virtual environment should be instructed in how to perform their dance. However, this can be a time-consuming and challenging task with the current Virtual Reality (VR) controllers. To address these challenges, we introduce an innovative choreography system leveraging Large Language Models (LLMs) to automate dance creation and presentation in VR. To achieve this goal, we first establish a corpus dataset comprising almost five thousand prompt phrases to describe fundamental movements and formation changes within dance formation choreography. We then leverage an LLM to understand these prompts, thereby comprehending the commands issued by users during the interaction. Based on the understanding provided by the LLM, these prompts are converted into primitive commands that our system can execute. This allows the virtual character to perform corresponding actions to complete the dance formation choreography. We conduct three user studies with 94 subjects to validate the proposed approach. Through user studies, we find that the proposed approach can help users generate high-quality ICH dances regarding human likeness, appropriateness, and operability. For the demo, please refer to this link .