Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integrating Skeleton Based Representations for Robust Yoga Pose Classification Using Deep Learning Models

Mohammed Mohiuddin, Syed Mohammod Minhaz Hossain, Sumaiya Khanam, Prionkar Barua, Aparup Barua, MD Tamim Hossain

TL;DR

This work introduces Yoga-16, a balanced 16-pose dataset, and a comprehensive benchmark comparing direct-image and skeleton-based inputs across three CNN architectures. Skeleton-based representations, especially MediaPipe skeletons with VGG16, achieve the highest accuracy (96.09%), supported by Grad-CAM interpretability and cross-validation evidence of strong generalization. The study demonstrates the practical value of skeletal pose representations for robust yoga pose classification and provides a framework for evaluating real-time fitness applications. Collective findings highlight the potential of skeleton-focused pipelines to improve accuracy, robustness, and explainability in digital health and wellness tools.

Abstract

Yoga is a popular form of exercise worldwide due to its spiritual and physical health benefits, but incorrect postures can lead to injuries. Automated yoga pose classification has therefore gained importance to reduce reliance on expert practitioners. While human pose keypoint extraction models have shown high potential in action recognition, systematic benchmarking for yoga pose recognition remains limited, as prior works often focus solely on raw images or a single pose extraction model. In this study, we introduce a curated dataset, 'Yoga-16', which addresses limitations of existing datasets, and systematically evaluate three deep learning architectures (VGG16, ResNet50, and Xception), using three input modalities (direct images, MediaPipe Pose skeleton images, and YOLOv8 Pose skeleton images). Our experiments demonstrate that skeleton-based representations outperform raw image inputs, with the highest accuracy of 96.09% achieved by VGG16 with MediaPipe Pose skeleton input. Additionally, we provide interpretability analysis using Grad-CAM, offering insights into model decision-making for yoga pose classification with cross-validation analysis.

Integrating Skeleton Based Representations for Robust Yoga Pose Classification Using Deep Learning Models

TL;DR

This work introduces Yoga-16, a balanced 16-pose dataset, and a comprehensive benchmark comparing direct-image and skeleton-based inputs across three CNN architectures. Skeleton-based representations, especially MediaPipe skeletons with VGG16, achieve the highest accuracy (96.09%), supported by Grad-CAM interpretability and cross-validation evidence of strong generalization. The study demonstrates the practical value of skeletal pose representations for robust yoga pose classification and provides a framework for evaluating real-time fitness applications. Collective findings highlight the potential of skeleton-focused pipelines to improve accuracy, robustness, and explainability in digital health and wellness tools.

Abstract

Yoga is a popular form of exercise worldwide due to its spiritual and physical health benefits, but incorrect postures can lead to injuries. Automated yoga pose classification has therefore gained importance to reduce reliance on expert practitioners. While human pose keypoint extraction models have shown high potential in action recognition, systematic benchmarking for yoga pose recognition remains limited, as prior works often focus solely on raw images or a single pose extraction model. In this study, we introduce a curated dataset, 'Yoga-16', which addresses limitations of existing datasets, and systematically evaluate three deep learning architectures (VGG16, ResNet50, and Xception), using three input modalities (direct images, MediaPipe Pose skeleton images, and YOLOv8 Pose skeleton images). Our experiments demonstrate that skeleton-based representations outperform raw image inputs, with the highest accuracy of 96.09% achieved by VGG16 with MediaPipe Pose skeleton input. Additionally, we provide interpretability analysis using Grad-CAM, offering insights into model decision-making for yoga pose classification with cross-validation analysis.

Paper Structure

This paper contains 32 sections, 28 equations, 13 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Benchmark Work Using Direct Image Input or Non-Skeletonization Approach
  4. Benchmark Work Using Skeletonization Approach
  5. Benchmark Work Using Transfer Learning Approach
  6. Methodology
  7. Problem Formulation
  8. Data collection and pre-processing
  9. Preliminaries
  10. Image Representation and Normalization
  11. Core Deep Learning Operations
  12. Pose Keypoints and Skeleton Representation
  13. Kalman Filter for Smoothing
  14. Training and Hyperparameter Tuning
  15. Extracting Human Pose Keypoints using Mediapipe Model
  16. ...and 17 more sections

Figures (13)

  • Figure 1: The Workflow of Yoga Pose Classification
  • Figure 2: Examples of issues observed in the Yoga-82 dataset: (a) multiple human subjects in one image, (b) low-quality image, (c) non-human subjects such as shapes, and (d) zoomed-out individual makes identifying the pose difficult.
  • Figure 3: Sample images from Yoga-16 Dataset
  • Figure 4: Keypoint Extraction Pipeline in MediaPipe
  • Figure 5: Sample Output Images of MediaPipe Human Pose Skeleton Extraction
  • ...and 8 more figures