Automatic Recognition of Gesture Qualities in Speech Delivery in VR-Work-in-Progress

Gesture use during public speaking can be analyzed as an aspect of speaker performance as well as an indicator of speaker emotions. Recognizing gesture quality rather than identifying specific gestures presents an underexplored challenge compared to traditional gesture recognition in virtual reality (VR). To that end, we use a VR headset and controllers to create a database of 162 five-second-long gestures. Next, we ask ten judges to evaluate the quality of gestures in three dimensions, i.e., dynamics, range, and between-hand distance, on a low-medium-high scale with average Fleiss Kappa 0.39 inter-rater agreement. A comparison of various classifiers revealed that the Support Vector Machine (SVM) yielded the best results, achieving classification accuracy of 70-85% for each quality dimension.