Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ViSTec: Video Modeling for Sports Technique Recognition and Tactical Analysis

Yuchen He, Zeqing Yuan, Yihong Wu, Liqi Cheng, Dazhen Deng, Yingcai Wu

TL;DR

ViSTec, a Video-based Sports Technique recognition model inspired by human cognition that synergizes sparse visual data with rich contextual insights, is proposed, which outperforms existing models by a significant margin.

Abstract

The immense popularity of racket sports has fueled substantial demand in tactical analysis with broadcast videos. However, existing manual methods require laborious annotation, and recent attempts leveraging video perception models are limited to low-level annotations like ball trajectories, overlooking tactics that necessitate an understanding of stroke techniques. State-of-the-art action segmentation models also struggle with technique recognition due to frequent occlusions and motion-induced blurring in racket sports videos. To address these challenges, We propose ViSTec, a Video-based Sports Technique recognition model inspired by human cognition that synergizes sparse visual data with rich contextual insights. Our approach integrates a graph to explicitly model strategic knowledge in stroke sequences and enhance technique recognition with contextual inductive bias. A two-stage action perception model is jointly trained to align with the contextual knowledge in the graph. Experiments demonstrate that our method outperforms existing models by a significant margin. Case studies with experts from the Chinese national table tennis team validate our model's capacity to automate analysis for technical actions and tactical strategies. More details are available at: https://ViSTec2024.github.io/.

ViSTec: Video Modeling for Sports Technique Recognition and Tactical Analysis

TL;DR

ViSTec, a Video-based Sports Technique recognition model inspired by human cognition that synergizes sparse visual data with rich contextual insights, is proposed, which outperforms existing models by a significant margin.

Abstract

The immense popularity of racket sports has fueled substantial demand in tactical analysis with broadcast videos. However, existing manual methods require laborious annotation, and recent attempts leveraging video perception models are limited to low-level annotations like ball trajectories, overlooking tactics that necessitate an understanding of stroke techniques. State-of-the-art action segmentation models also struggle with technique recognition due to frequent occlusions and motion-induced blurring in racket sports videos. To address these challenges, We propose ViSTec, a Video-based Sports Technique recognition model inspired by human cognition that synergizes sparse visual data with rich contextual insights. Our approach integrates a graph to explicitly model strategic knowledge in stroke sequences and enhance technique recognition with contextual inductive bias. A two-stage action perception model is jointly trained to align with the contextual knowledge in the graph. Experiments demonstrate that our method outperforms existing models by a significant margin. Case studies with experts from the Chinese national table tennis team validate our model's capacity to automate analysis for technical actions and tactical strategies. More details are available at: https://ViSTec2024.github.io/.
Paper Structure (20 sections, 8 equations, 4 figures, 1 table, 1 algorithm)

This paper contains 20 sections, 8 equations, 4 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Sports Action Recognition and Segmentation
  4. Racket Sports Data Mining and Analysis
  5. Data Descriptions and Notations
  6. Racket Sports Data
  7. Notations
  8. ViSTec
  9. Video Feature Modeling
  10. Stroke Segmentation
  11. Stroke Classification with Contextual Knowledge
  12. Inference of ViSTec
  13. Experiments
  14. Experiment Dataset
  15. Comparative Study
  16. ...and 5 more sections

Figures (4)

  • Figure 1: The framework of ViSTec. (A) is the stroke segmentation module. (B) is the $cls$ module, with segmented stroke features as input and probability distributions for each segment as output, as shown in (C). (D) is the $grh$ module for domain knowledge modeling. (E) shows the detail of video feature extractor.
  • Figure 2: Illustration of segmentation result for a sample from our dataset with the ground truth sequence "Serve, Short, Short, Topspin, Block".
  • Figure 3: Case 1: (A) displays visual features of the strokes from two Japanese players with t-SNE. (B) highlights techniques that share noticeable similarities. (C) shows the technique actions highlighted in (B).
  • Figure 4: Case 2: (A) shows the structure of a table tennis tactic, consisting of consecutive three strokes. (B) illustrates the scoring rate of consecutive three strokes. (C) illustrates the scoring rate using different techniques after "Serve, Short".