Leveraging MediaPipe and YOLO Keypoint Detection in Ensemble Approaches for Workout Pose Recognition

The identification and estimation of exercise poses has been a field of ongoing research in computer vision. Many deep learning architectures have demonstrated impressive performance, and as a result, much progress has been achieved in this area over the years. These developments highlight the enormous potential that exists at the nexus of fitness-related applications and computer vision, which applies to both images and videos. The complex task of workout image classification is addressed here using keypoint based classification, using YOLOv8 and MediaPipe to identify keypoints in the image and using those keypoints for classification. Then rigorous experiments were performed on a custom dataset designed to cover a wide range of poses—including differences in camera viewpoints, angles, and exercises. As part of this research, multiple algorithms were trained, including ensemble learning algorithms in an attempt to differentiate between the model performance of each algorithm. The effectiveness and limitations of different approaches have been measured in correctly identifying and classifying workout poses from 2D images. Analysis using multiple techniques is presented, where LightGBM with MediaPipe shows the best performance, achieving an accuracy of 95.95%.