Classifying Interpersonal Synchrony in VR Joint Actions: Human-Human versus Human-Bot Interactions

Abstract This study collected sensor data from VR goggles in pairs and examined interpersonal synchrony using a within-subject design. Participants positioned opposite each other in VR space judged object categories (kitchen vs. garage utensils) and responded with movements. We created an LSTM classification model using movement data from pairs instructed to interact either competitively (Game session) or synchronously (Collab session). The model achieved 93.2% accuracy (F1 score = 0.93.2) in classifying these interaction modes using only three-axis head acceleration data from the goggles. We then applied this within-subject trained model to Bot interaction sessions, where participants worked with Bot avatars under high or low accuracy conditions. Dynamic Time Warping (DTW) distance was used to quantify synchrony. Results showed that when movements were classified as synchronous, DTW distance was lower, indicating higher synchrony. A significant interaction effect was found between movement classification (Game/Collab) and Bot accuracy condition on DTW distance. Exploratory cross-subject validation showed reduced performance (F1 = 0.72), though head rotation features remained important across participants. These findings demonstrate that the classification model successfully captures interaction dynamics across different task conditions within subjects, and that synchrony with Bots is modulated by behavioral predictability.