OST: Efficient One-stream Network for 3D Single Object Tracking in Point Clouds

TL;DR

This work tackles 3D single object tracking in LiDAR point clouds, addressing the inefficiency and limited instance discrimination of Siamese trackers by introducing a one-stream architecture. The proposed Template-aware Transformer Module (TTM) and Multi-scale Feature Aggregation (MFA) fuse template information into search features and balance spatial with semantic cues, enabling robust tracking across seen and unseen categories with lower computational cost. Empirical results on KITTI and nuScenes show strong class-specific performance and notable gains in class-agnostic settings, supported by comprehensive ablations that validate design choices. The method offers practical benefits for real-time autonomous systems by combining efficient feature learning with template-guided discrimination.

Abstract

Although recent Siamese network-based trackers have achieved impressive perceptual accuracy for single object tracking in LiDAR point clouds, they usually utilized heavy correlation operations to capture category-level characteristics only, and overlook the inherent merit of arbitrariness in contrast to multiple object tracking. In this work, we propose a radically novel one-stream network with the strength of the instance-level encoding, which avoids the correlation operations occurring in previous Siamese network, thus considerably reducing the computational effort. In particular, the proposed method mainly consists of a Template-aware Transformer Module (TTM) and a Multi-scale Feature Aggregation (MFA) module capable of fusing spatial and semantic information. The TTM stitches the specified template and the search region together and leverages an attention mechanism to establish the information flow, breaking the previous pattern of independent \textit{extraction-and-correlation}. As a result, this module makes it possible to directly generate template-aware features that are suitable for the arbitrary and continuously changing nature of the target, enabling the model to deal with unseen categories. In addition, the MFA is proposed to make spatial and semantic information complementary to each other, which is characterized by reverse directional feature propagation that aggregates information from shallow to deep layers. Extensive experiments on KITTI and nuScenes demonstrate that our method has achieved considerable performance not only for class-specific tracking but also for class-agnostic tracking with less computation and higher efficiency.

Figures (9)

• Figure 1: Comparison between Siamese-based and one-stream frameworks attached their visualization results in case of interference from other similar targets. (a) The basic process of classical Siamese methods, e.g. P2B, V2B. (b) The novel approach with one-stream frameworks. (c) The visualization result of V2B and our method, where ours captures the target pedestrian while V2B is misled by the interferes.
• Figure 2: The overview of the proposed OST. After concatenating the template and the search region, we send it into the one-stream network consisting of local encoding module, template-aware Transformer module, and multi-scale feature aggregation module. In this way, we can generate template-aware features of search region. Afterwards, following the feature augmentation by the segmentation prior, we voxelized feature tensor for proposal generation. The overall process is at the top of this figure, and the module details are at the bottom.
• Figure 3: The architecture of our Transformer encoder and two distinctive aggregation strategies. (a) The basic process of Transformer in our method. (b) The common steps for multi-scale aggregation. (c) Inverse sampling strategy for multi-scale aggregation with our template-aware Transformer module (TTM).
• Figure 4: The architecture of feature augmentation and detection head. The feature augmentation is guided by the segmentation head which provides interior and exterior cues. The detection head is used to regress the bounding box following V2B.
• Figure 5: The visualization of the feature distribution of the target and background via T-SNE van2008visualizing method. (a) The visualizations of search features from class-specific testing; (b) The visualizations of search features from class-agnostic testing. (a)/(b) We visualize the search region point cloud features from four different frames in KITTI car dataset. The subfigures in each row are from the same frame, and the subfigures in each column are from the same method. The target points are plotted by blue, and the background points plotted by red.