What question did this study set out to answer?

The research aims to develop an automated system to evaluate human actions in 3D through motion capture in a VR environment.

April 1, 2026Open Access

Spatio-temporal graph autoencoder for automated evaluation of human actions in 3D in immersive VR-based training for archaeologists

Key Points

The research aims to develop an automated system to evaluate human actions in 3D through motion capture in a VR environment.
Built a VR system that integrates motion tracking using Xsens MVN suit.
Constructed graphs to represent key movement features for analysis.
Employed an autoencoder using graph convolutional networks and LSTM for spatial and temporal modeling.
Utilized semi-supervised clustering to classify movements based on predefined centroids.
Provided real-time corrective feedback by reconstructing movements.
The system effectively recognized and analyzed movement patterns.
Live tests showed the model's capability to provide corrective feedback on movements.
Participants demonstrated improved accuracy in actions during training sessions.

Abstract

Motion capture and analysis is a field of application in constant development, and improvement can be seen in both the field of advanced models and new devices. Its integration with Virtual Reality (VR) can further expand the field of application. Here, a system is built to train all those movements that require precision and accuracy in body movements, using VR to immerse the user in the relevant environment and the Xsens MVN (suit) to track and analyze the movements. A model processes motion data by constructing graphs where nodes represent key movement features. These are input to an Autoencoder (AE), AEforGraph, composed of a Graph Convolutional Network (GCN) for spatial dependencies and a Long-Short Term Memory (LSTM) network for temporal modeling. The encoded representations undergo Semi-Supervised Clustering to classify movements based on their similarity to predefined centroids representing correct execution. The decoder reconstructs the movement to highlight deviations and provide real-time corrective feedback. Live tests confirm the system’s effectiveness in recognizing and analyzing movement patterns, making it a valuable tool for training applications.

Bookmark

View Full Paper