A Real‐Time VIMS Prediction Model Based on Short Long‐Term Visual Motion Features

ABSTRACT Visually induced motion sickness (VIMS) is one of the major obstacles to the broader adoption of virtual reality (VR) technology. As visual‐vestibular conflict is considered a major factor contributing to VIMS, real‐time prediction from visual cues remains crucial for timely intervention and for improving user experience. In this paper, we present a real‐time VIMS prediction model that utilizes a hybrid architecture of a 3D Convolutional Neural Network and a Long Short‐Term Memory network to capture both short and long‐term features. We used low‐resolution frames and replaced optical flow with frame difference maps and lateral/vertical displacement components to reduce data complexity and accelerate model inference. Experimental results on two datasets demonstrate that our model outperforms existing vision‐based approaches, such as VR‐SP and VR‐SA, with an RMSE of 3.77 and a PLCC of 0.935. Importantly, the average processing time for each 108‐frame video is less than 0.83 s. In addition, the method's real‐time VIMS prediction capability has been verified in a Unity3D‐based VR environment, with an end‐to‐end latency of 192 ms. These results highlight the model's advantages in terms of both efficiency and accuracy, making it a promising solution for VIMS‐aware VR applications.