STO-CVAE: state transition-oriented conditional variational autoencoder for data augmentation in disability classification

Abstract The class imbalance problem occurs when there is an unequal distribution of classes in a dataset and is a significant issue in various artificial intelligence applications. This study focuses on the severe multiclass imbalance problem of human activity recognition in rehabilitation exercises for people with disabilities. To overcome this problem, we present a novel human action-centric augmentation method for human skeleton-based pose estimation. This study proposes the state transition-oriented conditional variational autoencoder (STO-CVAE) to capture action patterns in repeated exercises. The proposed approach generates action samples by capturing temporal information of human skeletons to improve the identification of minority disability classes. We conducted experimental studies with a real-world dataset gathered from rehabilitation exercises and confirmed the superiority and effectiveness of the proposed method. Specifically, all investigated classifiers (i.e., random forest, support vector machine, extreme gradient boosting, light gradient boosting machine, and TabNet) trained with the proposed augmentation method outperformed the models trained without augmentation in terms of the F 1-score and accuracy, with F 1-score showing the most improvement. Overall, the prediction accuracy of most classes was improved; in particular, the prediction accuracy of the minority classes was greatly improved. Hence, the proposed STO-CVAE can be used to improve the accuracy of disability classification in the field of physical medicine and rehabilitation and to provide suitable personal training and rehabilitation exercise programs.