Enhancing Yoga Pose Recognition through a Multimodal Hybrid Approach with Attention Mechanism and Late Fusion on Yoga For All-30

Context: The project focuses on yoga pose recognition, a critical task in computer vision and AI, to enhance physical and mental well-being by improving posture analysis. It addresses challenges such as the lack of benchmark datasets and limitations in both landmark-augmented and landmark-free methods. Aims: 1. Developed a hybrid model combining landmark-based and landmark-free techniques for accurate yoga pose recognition 2. Introduced YFA-30, a comprehensive dataset with 30 yoga poses to benchmark pose recognition models 3. Conducted a comparative analysis of existing models with the proposed hybrid model. Settings and Design: The project operates within the domain of computer vision and artificial intelligence, specifically focusing on yoga pose recognition. It involves a controlled computational environment for training, validating, and testing deep learning models using datasets of yoga poses. The settings emphasize creating a system capable of real-time pose recognition and correction for practical applications in fitness, health, and education. Methods: • Hybrid Model: Combines MoveNet (landmark detection) and ResNet50 (image feature extraction) using a scaled dot-product attention mechanism and late fusion • Statistical Evaluation: Analyzes accuracy, loss, and performance metrics across multiple datasets (YFA-30, YFA-20, YFA-10, Kaggle, and Google CG) • Comparative Study: Benchmarks against models such as Mediapipe, YOLO-V8x, and Xception • Visualization: Uses Grad-CAM heatmaps for feature extraction insights. Material: Datasets: • YFA-30: Newly introduced dataset with 30 yoga poses with approximately 8000 images • Kaggle Dataset: Small dataset with 5 yoga poses • Google CG Dataset: Synthetic images of 5 yoga poses. Models: • Landmark-based: MoveNet, Mediapipe, and YOLO-V8x • Landmark-free: ResNet50, VGG16, EfficientNetB0, and Xception. Statistical Analysis Used: The statistical analyses used in your yoga pose recognition project include: 1. Performance Metrics: Accuracy, loss (training, validation, and test), and loss curves across datasets (YFA-30, YFA-20, YFA-10, Kaggle, and Google CG) to evaluate model efficiency 2. Comparative Analysis: Benchmarking the hybrid model (MoveNet + ResNet50) against other models such as Mediapipe, YOLO-V8x-pose, and landmark-free models (e.g., VGG16 and Xception) 3. Cross-Validation: Testing on YFA dataset subsets to ensure robustness across data variability 4. Attention Mechanism: Scaled dot-product attention and late fusion to integrate and prioritize relevant features and landmarks 5. Visual Analysis: Heatmaps (Grad-CAM) to interpret keypoint detection and feature extraction 6. Dataset Testing: Evaluation on synthetic and real-world datasets to ensure adaptability. These analyses validate the hybrid model’s performance and robustness.