Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TeamTrack: A Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos

Atom Scott, Ikuma Uchida, Ning Ding, Rikuhei Umemoto, Rory Bunker, Ren Kobayashi, Takeshi Koyama, Masaki Onishi, Yoshinari Kameda, Keisuke Fujii

TL;DR

TeamTrack provides a large-scale, full-pitch MOT benchmark for team sports (soccer, basketball, handball) with multi-view coverage (top-view and side-view) to address appearance-based tracking challenges. It includes over $2\times 10^5$ frames and over $4\times 10^6$ bounding boxes across sports, captured from two viewpoints, and offers data collection, annotation, and baseline evaluations for object detection, trajectory forecasting, and MOT. The dataset exposes high within-team appearance similarity, complex motion patterns, and substantial annotation effort, enabling robust evaluation of modern tracking algorithms and multi-view fusion approaches. This dataset enables more robust tracking in sports and can catalyze advances in multi-view tracking and sports analytics.

Abstract

Multi-object tracking (MOT) is a critical and challenging task in computer vision, particularly in situations involving objects with similar appearances but diverse movements, as seen in team sports. Current methods, largely reliant on object detection and appearance, often fail to track targets in such complex scenarios accurately. This limitation is further exacerbated by the lack of comprehensive and diverse datasets covering the full view of sports pitches. Addressing these issues, we introduce TeamTrack, a pioneering benchmark dataset specifically designed for MOT in sports. TeamTrack is an extensive collection of full-pitch video data from various sports, including soccer, basketball, and handball. Furthermore, we perform a comprehensive analysis and benchmarking effort to underscore TeamTrack's utility and potential impact. Our work signifies a crucial step forward, promising to elevate the precision and effectiveness of MOT in complex, dynamic settings such as team sports. The dataset, project code and competition is released at: https://atomscott.github.io/TeamTrack/.

TeamTrack: A Dataset for Multi-Sport Multi-Object Tracking in Full-pitch Videos

TL;DR

TeamTrack provides a large-scale, full-pitch MOT benchmark for team sports (soccer, basketball, handball) with multi-view coverage (top-view and side-view) to address appearance-based tracking challenges. It includes over frames and over bounding boxes across sports, captured from two viewpoints, and offers data collection, annotation, and baseline evaluations for object detection, trajectory forecasting, and MOT. The dataset exposes high within-team appearance similarity, complex motion patterns, and substantial annotation effort, enabling robust evaluation of modern tracking algorithms and multi-view fusion approaches. This dataset enables more robust tracking in sports and can catalyze advances in multi-view tracking and sports analytics.

Abstract

Multi-object tracking (MOT) is a critical and challenging task in computer vision, particularly in situations involving objects with similar appearances but diverse movements, as seen in team sports. Current methods, largely reliant on object detection and appearance, often fail to track targets in such complex scenarios accurately. This limitation is further exacerbated by the lack of comprehensive and diverse datasets covering the full view of sports pitches. Addressing these issues, we introduce TeamTrack, a pioneering benchmark dataset specifically designed for MOT in sports. TeamTrack is an extensive collection of full-pitch video data from various sports, including soccer, basketball, and handball. Furthermore, we perform a comprehensive analysis and benchmarking effort to underscore TeamTrack's utility and potential impact. Our work signifies a crucial step forward, promising to elevate the precision and effectiveness of MOT in complex, dynamic settings such as team sports. The dataset, project code and competition is released at: https://atomscott.github.io/TeamTrack/.
Paper Structure (14 sections, 3 equations, 9 figures, 6 tables)

This paper contains 14 sections, 3 equations, 9 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. TeamTrack Dataset
  4. Data Collection
  5. Annotation and Labeling
  6. Appearance Similarity
  7. Motion Patterns
  8. Experiments and Evaluation
  9. Object Detection
  10. Trajectory Forecasting
  11. Multiple Object Tracking
  12. Conclusion
  13. Acknowledgements
  14. Change Log

Figures (9)

  • Figure 1: Example of our TeamTrack dataset. We provide (a) top-view and (b) side-view in soccer, (c) top-view and, (d)/(e) two side-view videos in basketball, and (f) a single side-view video in handball.
  • Figure 2: Screenshot of Labelbox during the annotation process, illustrating manual adjustments to automated tracking and interpolation annotations.
  • Figure 3: Cosine distance of re-ID features: TeamTrack compared to DanceTrack, MOT17, and MOT20.
  • Figure 4: Cosine distance of re-ID features within TeamTrack datasets.
  • Figure 5: Visualization of re-ID features from various videos in our TeamTrack dataset using t-SNE. Objects are color-coded for consistent identification. The bounding box from the first frame is superimposed on the corresponding re-ID feature for contextualization.
  • ...and 4 more figures