Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tracking Transforming Objects: A Benchmark

You Wu, Yuelong Wang, Yaxin Liao, Fuliang Wu, Hengzhou Ye, Shuiwang Li

TL;DR

This work addresses tracking transforming objects, where objects change appearance, context, or even category over time. It introduces DTTO, a dedicated benchmark comprising 100 sequences (~9.3K frames) across 11 categories and six transformation types, with frame-wise bounding boxes and attribute annotations. The authors evaluate 20 SOTA trackers spanning CNN-based, CNN-Transformer-based, and Transformer-based architectures on DTTO using one-pass evaluation with PRC and SUC, establishing baselines and insights. Results show Transformer-based trackers often lead, but certain transformations remain challenging, underscoring the need for transformation-aware tracking, and DTTO is positioned to spur progress in robust perception for dynamic environments.

Abstract

Tracking transforming objects holds significant importance in various fields due to the dynamic nature of many real-world scenarios. By enabling systems accurately represent transforming objects over time, tracking transforming objects facilitates advancements in areas such as autonomous systems, human-computer interaction, and security applications. Moreover, understanding the behavior of transforming objects provides valuable insights into complex interactions or processes, contributing to the development of intelligent systems capable of robust and adaptive perception in dynamic environments. However, current research in the field mainly focuses on tracking generic objects. In this study, we bridge this gap by collecting a novel dedicated Dataset for Tracking Transforming Objects, called DTTO, which contains 100 sequences, amounting to approximately 9.3K frames. We provide carefully hand-annotated bounding boxes for each frame within these sequences, making DTTO the pioneering benchmark dedicated to tracking transforming objects. We thoroughly evaluate 20 state-of-the-art trackers on the benchmark, aiming to comprehend the performance of existing methods and provide a comparison for future research on DTTO. With the release of DTTO, our goal is to facilitate further research and applications related to tracking transforming objects.

Tracking Transforming Objects: A Benchmark

TL;DR

This work addresses tracking transforming objects, where objects change appearance, context, or even category over time. It introduces DTTO, a dedicated benchmark comprising 100 sequences (~9.3K frames) across 11 categories and six transformation types, with frame-wise bounding boxes and attribute annotations. The authors evaluate 20 SOTA trackers spanning CNN-based, CNN-Transformer-based, and Transformer-based architectures on DTTO using one-pass evaluation with PRC and SUC, establishing baselines and insights. Results show Transformer-based trackers often lead, but certain transformations remain challenging, underscoring the need for transformation-aware tracking, and DTTO is positioned to spur progress in robust perception for dynamic environments.

Abstract

Tracking transforming objects holds significant importance in various fields due to the dynamic nature of many real-world scenarios. By enabling systems accurately represent transforming objects over time, tracking transforming objects facilitates advancements in areas such as autonomous systems, human-computer interaction, and security applications. Moreover, understanding the behavior of transforming objects provides valuable insights into complex interactions or processes, contributing to the development of intelligent systems capable of robust and adaptive perception in dynamic environments. However, current research in the field mainly focuses on tracking generic objects. In this study, we bridge this gap by collecting a novel dedicated Dataset for Tracking Transforming Objects, called DTTO, which contains 100 sequences, amounting to approximately 9.3K frames. We provide carefully hand-annotated bounding boxes for each frame within these sequences, making DTTO the pioneering benchmark dedicated to tracking transforming objects. We thoroughly evaluate 20 state-of-the-art trackers on the benchmark, aiming to comprehend the performance of existing methods and provide a comparison for future research on DTTO. With the release of DTTO, our goal is to facilitate further research and applications related to tracking transforming objects.
Paper Structure (13 sections, 7 figures, 3 tables)

This paper contains 13 sections, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Datasets for Generic Visual Object Tracking
  4. Visual Tracking Algorithms
  5. The Proposed DTTO Dataset
  6. Data Collection
  7. Annotation
  8. Attributes
  9. Experiments
  10. Evaluation Metrics
  11. Evaluated Trackers
  12. Evaluation Results
  13. Conclusion

Figures (7)

  • Figure 1: The examples of generic object tracking from GOT-10k huang2019got, LaSOT fan2019lasot, and TrackingNet muller2018trackingnet (a) and tracking transforming objects from our DTTO (b).
  • Figure 2: Statistics of DTTO: on the left, the distribution of transformations, and on the right, the co-occurrence statistics between the transformations and object categories.
  • Figure 3: Visual representation of annotation examples for three types (i.e., bloom, collide, and transform) within the proposed DTTO.
  • Figure 4: Distribution of sequences per attribute.
  • Figure 5: The overall performance of 20 SOTA trackers are evaluated on DTTO. Precision and success rate, as defined by one-pass evaluation (OPE)wu2013online, are employed for evaluation.
  • ...and 2 more figures