Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

3D Pose-Based Temporal Action Segmentation for Figure Skating: A Fine-Grained and Jump Procedure-Aware Annotation Approach

Ryota Tanaka, Tomohiro Suzuki, Keisuke Fujii

TL;DR

This study created the FS-Jump3D dataset of complex and dynamic figure skating jumps using optical markerless motion capture and proposed a new fine-grained figure skating jump TAS dataset annotation method with which TAS models can learn jump procedures.

Abstract

Understanding human actions from videos is essential in many domains, including sports. In figure skating, technical judgments are performed by watching skaters' 3D movements, and its part of the judging procedure can be regarded as a Temporal Action Segmentation (TAS) task. TAS tasks in figure skating that automatically assign temporal semantics to video are actively researched. However, there is a lack of datasets and effective methods for TAS tasks requiring 3D pose data. In this study, we first created the FS-Jump3D dataset of complex and dynamic figure skating jumps using optical markerless motion capture. We also propose a new fine-grained figure skating jump TAS dataset annotation method with which TAS models can learn jump procedures. In the experimental results, we validated the usefulness of 3D pose features as input and the fine-grained dataset for the TAS model in figure skating. FS-Jump3D Dataset is available at https://github.com/ryota-skating/FS-Jump3D.

3D Pose-Based Temporal Action Segmentation for Figure Skating: A Fine-Grained and Jump Procedure-Aware Annotation Approach

TL;DR

This study created the FS-Jump3D dataset of complex and dynamic figure skating jumps using optical markerless motion capture and proposed a new fine-grained figure skating jump TAS dataset annotation method with which TAS models can learn jump procedures.

Abstract

Understanding human actions from videos is essential in many domains, including sports. In figure skating, technical judgments are performed by watching skaters' 3D movements, and its part of the judging procedure can be regarded as a Temporal Action Segmentation (TAS) task. TAS tasks in figure skating that automatically assign temporal semantics to video are actively researched. However, there is a lack of datasets and effective methods for TAS tasks requiring 3D pose data. In this study, we first created the FS-Jump3D dataset of complex and dynamic figure skating jumps using optical markerless motion capture. We also propose a new fine-grained figure skating jump TAS dataset annotation method with which TAS models can learn jump procedures. In the experimental results, we validated the usefulness of 3D pose features as input and the fine-grained dataset for the TAS model in figure skating. FS-Jump3D Dataset is available at https://github.com/ryota-skating/FS-Jump3D.
Paper Structure (21 sections, 5 figures, 6 tables)

This paper contains 21 sections, 5 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Proposed Framework
  4. FS-Jump3D dataset
  5. Optical Markerless Motion Capture
  6. Setup
  7. Participants
  8. Capturing Conditions
  9. Training 3D Pose Estimation Model with FS-Jump3D
  10. Fine-Grained and Jump Procedure-Aware Annotation
  11. Figure Skating Jump TAS
  12. Experiments
  13. Datasets
  14. FS-Jump3D Dataset
  15. Figure Skating TAS Dataset
  16. ...and 6 more sections

Figures (5)

  • Figure 1: The overview of our proposed method. First, we created the FS-Jump3D dataset of complex and dynamic figure skating jumps using optical markerless motion capture. Second, we propose a new fine-grained figure skating jump TAS dataset annotation (the difference from MCFS is shown in the figure). Lastly, we estimate the 3D poses from the broadcast video dataset with DWPose DWPose and MotionAGFormer motionagformer2024, and perform TAS with FACT FACT on the dataset annotated with detailed figure skating jump procedures.
  • Figure 2: Layout of 12 high-speed cameras on the ice skating rink.
  • Figure 3: Examples of the variety of shooting angles in figure skating broadcast footage.
  • Figure 4: The number of occurrences of each jump in all 371 performance videos we annotated.
  • Figure 5: Visualization of estimation results with and without the FS-Jump3D dataset for training a 3D pose estimation model.