Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking

Haiji Liang, Ruize Han

TL;DR

A new large-scale benchmark for open-vocabulary multi-object tracking namely OVT-B, which contains 1,048 categories of objects and 1,973 videos with 637,608 bounding box annotations, which is much larger than the sole open-vocabulary tracking dataset.

Abstract

Open-vocabulary object perception has become an important topic in artificial intelligence, which aims to identify objects with novel classes that have not been seen during training. Under this setting, open-vocabulary object detection (OVD) in a single image has been studied in many literature. However, open-vocabulary object tracking (OVT) from a video has been studied less, and one reason is the shortage of benchmarks. In this work, we have built a new large-scale benchmark for open-vocabulary multi-object tracking namely OVT-B. OVT-B contains 1,048 categories of objects and 1,973 videos with 637,608 bounding box annotations, which is much larger than the sole open-vocabulary tracking dataset, i.e., OVTAO-val dataset (200+ categories, 900+ videos). The proposed OVT-B can be used as a new benchmark to pave the way for OVT research. We also develop a simple yet effective baseline method for OVT. It integrates the motion features for object tracking, which is an important feature for MOT but is ignored in previous OVT methods. Experimental results have verified the usefulness of the proposed benchmark and the effectiveness of our method. We have released the benchmark to the public at https://github.com/Coo1Sea/OVT-B-Dataset.

OVT-B: A New Large-Scale Benchmark for Open-Vocabulary Multi-Object Tracking

TL;DR

A new large-scale benchmark for open-vocabulary multi-object tracking namely OVT-B, which contains 1,048 categories of objects and 1,973 videos with 637,608 bounding box annotations, which is much larger than the sole open-vocabulary tracking dataset.

Abstract

Open-vocabulary object perception has become an important topic in artificial intelligence, which aims to identify objects with novel classes that have not been seen during training. Under this setting, open-vocabulary object detection (OVD) in a single image has been studied in many literature. However, open-vocabulary object tracking (OVT) from a video has been studied less, and one reason is the shortage of benchmarks. In this work, we have built a new large-scale benchmark for open-vocabulary multi-object tracking namely OVT-B. OVT-B contains 1,048 categories of objects and 1,973 videos with 637,608 bounding box annotations, which is much larger than the sole open-vocabulary tracking dataset, i.e., OVTAO-val dataset (200+ categories, 900+ videos). The proposed OVT-B can be used as a new benchmark to pave the way for OVT research. We also develop a simple yet effective baseline method for OVT. It integrates the motion features for object tracking, which is an important feature for MOT but is ignored in previous OVT methods. Experimental results have verified the usefulness of the proposed benchmark and the effectiveness of our method. We have released the benchmark to the public at https://github.com/Coo1Sea/OVT-B-Dataset.

Paper Structure

This paper contains 14 sections, 4 equations, 6 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. OVT Benchmark
  4. Data Collection
  5. Dataset Annotation
  6. Dataset Statistics and Comparison
  7. Dataset Attributes
  8. Metrics
  9. OVTrack+: A New Baseline
  10. Experiments
  11. Comparison with State-of-The-Art Methods
  12. In-depth Experimental Analysis
  13. Discussion
  14. Conclusion

Figures (6)

  • Figure 1: Comparison of OV-TAO-val and OVT-B.
  • Figure 2: Word cloud of OVT-B categories.
  • Figure 3: Comparison of OV-TAO-val with OVT-B in scope.
  • Figure 4: Screenshots of annotations of OV-TAO-val and OVT-B.
  • Figure 5: Ratio of videos with attributes.
  • ...and 1 more figures