Creating immersive experiences based on intangible cultural heritage

Cultural heritage is a lot more than monuments and collections of objects. In addition, it includes traditions and living expressions inherited from our ancestors and passed on to our descendants, such as songs, performing arts, social practices, rituals, festive events, knowledge and practices concerning nature, or the knowledge and skills to produce traditional crafts. These elements constitute the Intangible Cultural Heritage (ICH). Safeguarding of ICH is an important factor in maintaining social coherence along with cultural diversity, but it also is a challenge in modern societies. The digitization of ICH, with the transcription of its artifacts, is already becoming a springboard for its preservation and re-use, but intangible heritage has more to offer than a library of content. Lately, the advancements in extended reality interactive technologies and in artificial intelligence break new ground in understanding and sensing intangible heritage. In this paper, we present the CHROMATA platform which aims at facilitating the creation of novel immersive experiences to sustain ICH. To achieve this, we are developing a web platform that provides the content retrieval from online sources and cultural institutions; the AI services for multimedia analysis, e.g., 3D pose estimation, folklore dance recognition, and textual analysis; the authoring interface where the non-developers designers can create their virtual experience. The various data (images, video, text, audio, and 3D models) are analysed using state-of-the-art methods for dance recognition and visualization, Laban Movement Analysis and Labanotation generation, textual sentiment analysis and text generation. Moreover, 3D reconstruction techniques are employed to accurately capture places of interest, as well as human activities to create the related virtual spaces. The CHROMATA platform aims to boost the revival of intangible cultural heritage via easing the creation of more complete and enlightening immersive experiences. The platform development is driven and validated by real use case scenarios in which designers, creators and historians are building virtual experiences of customs and dances of Greek culture.