What question did this study set out to answer?

The aim is to develop a multimodal interactive narrative model that adapts to audience behavior and preferences using advanced AI techniques.

July 23, 2026Discover Artificial Intelligence0 citationsOpen Access

Multimodal interactive narration with virtual reality and exhibition and transformer-based audience behavior mining and plot adaptive algorithm

QWQingchan Wang

Key Points

The aim is to develop a multimodal interactive narrative model that adapts to audience behavior and preferences using advanced AI techniques.
Designed a three-tier multimodal architecture for perception fusion, behavior mining, and plot generation.
Employed a transformer encoder for integrating visual, auditory, and interactive data.
Utilized a composite reward function for generating personalized narratives based on audience data.
The proposed model outperformed mainstream baselines in key performance indicators.
Experimental validation showed effectiveness of core modules through ablation analysis.
Demonstrated potential for intelligent and personalized exhibition experiences.

Abstract

The progress of augmented reality (AR) and virtual reality (VR) technology has promoted the development of the virtual-real integration. Because the traditional fixed narrative mode is difficult to adapt to the individual differences of the audience, it leads to insufficient immersion and personalized experience. The existing methods still have limitations in multimodal data fusion, audience intention analysis and dynamic plot generation. Therefore, this study proposes a multimodal fusion interactive narrative model based on Transformer model, and constructs a three-tier architecture of multimodal perception fusion, audience behavior mining and adaptive plot generation. A cross-modal Transformer encoder is designed to deeply integrate visual, auditory, position trajectory and interactive log data to generate high-quality unified representation. The converter decoder is used to model the behavior sequence, perform macro behavior classification, interest regression and understanding estimation in parallel. A comprehensive behavior intention vector is constructed. The audience state is comprehensively captured. Combining the framework of Gated Recurrent Unit and Proximal Policy Optimization, a coherent, personalized and low-repetition plot is dynamically generated through a composite reward function. The final experimental results show that the model is significantly better than the mainstream baseline in indicators, and the ablation experiment verifies the effectiveness of each core module. The overall research can provide a complete technical solution for realizing intelligent and personalized exhibition experience, and has a good application prospect.

Ask AI

Helpful

Bookmark

View Full Paper