SCATR: Mitigating New Instance Suppression in LiDAR-based Tracking-by-Attention via Second Chance Assignment and Track Query Dropout

TL;DR

SCATR is introduced, a novel LiDAR-based TBA model designed to address this fundamental challenge systematically and achieves state-of-the-art performance among LiDAR-based TBA methods, outperforming previous works by 7.6\% AMOTA and successfully bridging the long-standing performance gap between LiDAR-based TBA and TBD methods.

Abstract

LiDAR-based tracking-by-attention (TBA) frameworks inherently suffer from high false negative errors, leading to a significant performance gap compared to traditional LiDAR-based tracking-by-detection (TBD) methods. This paper introduces SCATR, a novel LiDAR-based TBA model designed to address this fundamental challenge systematically. SCATR leverages recent progress in vision-based tracking and incorporates targeted training strategies specifically adapted for LiDAR. Our work's core innovations are two architecture-agnostic training strategies for TBA methods: Second Chance Assignment and Track Query Dropout. Second Chance Assignment is a novel ground truth assignment that concatenates unassigned track queries to the proposal queries before bipartite matching, giving these track queries a second chance to be assigned to a ground truth object and effectively mitigating the conflict between detection and tracking tasks inherent in tracking-by-attention. Track Query Dropout is a training method that diversifies supervised object query configurations to efficiently train the decoder to handle different track query sets, enhancing robustness to missing or newborn tracks. Experiments on the nuScenes tracking benchmark demonstrate that SCATR achieves state-of-the-art performance among LiDAR-based TBA methods, outperforming previous works by 7.6\% AMOTA and successfully bridging the long-standing performance gap between LiDAR-based TBA and TBD methods. Ablation studies further validate the effectiveness and generalization of Second Chance Assignment and Track Query Dropout. Code can be found at the following link: \href{https://github.com/TRAILab/SCATR}{https://github.com/TRAILab/SCATR}

Figures (4)

• Figure 1: a) In regular tracking-by-attention, when a track query is assigned to the vehicle, related proposal queries are suppressed, leading to sparse supervision of proposal queries, only from newborn objects ($PQ\;Z$). b) We propose Track Query Dropout, in which assigned track queries may not be propagated to subsequent frames. This provides proposal queries, such as $PQ\; Y$, positive supervision with the now unassigned car, conditioned on the absence of a related track query. c) Second Chance Assignment allows both unassigned track queries, like $TQ\;B$, to be assigned to unmatched ground truth objects.
• Figure 2: Architecture diagram of SCATR. Given BEV features of the scene and a set of anchor proposal queries, a detection decoder performs an initial detection of objects in the scene. The top-$N_{PQ}$ proposal queries are concatenated with the track queries from the previous frame and passed to the track decoder. The top-$N_{TQ}$ output queries are propagated to the next frame as track queries. Second Chance Assignment allows both track queries and proposal queries to initialize tracks.
• Figure 3: Creating an auxiliary query group by dropping the track query assigned to object $B$ trains the decoder to only suppress the potential proposal query for object $B$ conditioned on whether the assigned track query is present.
• Figure 4: In Second Chance Assignment, unassigned track queries are also considered when matching unassigned ground truth objects, and may take on new track IDs during training.