A New Dataset and a Distractor-Aware Architecture for Transparent Object Tracking

TL;DR

This paper addresses the gap in transparent object tracking by introducing Trans2k, the first large-scale, renderable training dataset with bounding boxes, segmentation masks, and distractor annotations, enabling deep trackers to learn background-dependent appearances. It also proposes DiTra, a distractor-aware architecture that separates localization from target identification through two parallel branches (pose-aware and distractor-aware) guided by a transformer-based image encoding module, with a two-phase training regime. Trans2k-trained models show consistent gains across major trackers on TOTB, with DiTra achieving state-of-the-art performance for transparent objects and strong generalization to opaque tracking. The work provides a practical framework for training and deploying robust transparent-object trackers and suggests avenues for long-term re-detection and specialized feature extraction to address remaining failure modes.

Abstract

Performance of modern trackers degrades substantially on transparent objects compared to opaque objects. This is largely due to two distinct reasons. Transparent objects are unique in that their appearance is directly affected by the background. Furthermore, transparent object scenes often contain many visually similar objects (distractors), which often lead to tracking failure. However, development of modern tracking architectures requires large training sets, which do not exist in transparent object tracking. We present two contributions addressing the aforementioned issues. We propose the first transparent object tracking training dataset Trans2k that consists of over 2k sequences with 104,343 images overall, annotated by bounding boxes and segmentation masks. Standard trackers trained on this dataset consistently improve by up to 16%. Our second contribution is a new distractor-aware transparent object tracker (DiTra) that treats localization accuracy and target identification as separate tasks and implements them by a novel architecture. DiTra sets a new state-of-the-art in transparent object tracking and generalizes well to opaque objects.

Figures (12)

• Figure 1: A tracker STARK* trained on opaque objects fails on transparent objects, while its performance remarkably improves after training on the proposed Trans2k dataset (first and second row). Both versions, however, fail in presence of visual distractors (third and fourth row), while the proposed DiTra comfortably tracks due to the new distractor-aware visual model.
• Figure 2: Trans2k attribute levels for "Transparency", "Motion blur", "Partial occlusion", "Distractor" (binary), "Target motion" (four control points) and "Rotation".
• Figure 3: A diverse set of object instances used in rendering Trans2k sequences.
• Figure 4: Targets in Trans2k are annotated by axis-aligned bounding boxes (first row) or by segmentation masks (second and third rows). The dataset also contains annotated distractors (third row).
• Figure 5: Overview of the proposed DiTra architecture. Features are first extracted from the search region and from a set of templates by Image encoding module (IEM). These features are then processed by two parallel branches generating pose-aware and distractor-aware features ($\mathbf{f}_{t}^{POS}$ and $\mathbf{f}_{t}^{DIS}$). Both features are summed together and processed by a bounding box prediction head to predict the target bounding box $\mathbf{B}_t$. Localization confidence score $s_t$ is estimated using the score prediction module (SPM).