Small Object Tracking in LiDAR Point Cloud: Learning the Target-awareness Prototype and Fine-grained Search Region

TL;DR

This work tackles small object tracking in LiDAR point clouds, where sparse foreground points and convolution-induced feature erosion hinder accurate localization. It introduces a Siamese framework augmented with Target-awareness Prototype Mining (TAPM) and Regional Grid Subdivision (RGS): TAPM reconstructs a target-aware prototype from the full feature space using a masked decoder and self-attention, while RGS upsamples the BEV feature map with a ViT-based transformer and pixel shuffle to recover fine-grained details. The method is trained with a composite loss including focal, regression, and Chamfer-distance terms to align prototype interpolation with the true target, and evaluated on KITTI and nuScenes with scaling experiments to simulate small objects. Results show improved small-object tracking performance, particularly for pedestrians, and robustness to scale changes, while maintaining performance on normal-sized targets. The approach advances practical LiDAR perception where reliably locating small objects is critical for navigation and safety.

Abstract

Single Object Tracking in LiDAR point cloud is one of the most essential parts of environmental perception, in which small objects are inevitable in real-world scenarios and will bring a significant barrier to the accurate location. However, the existing methods concentrate more on exploring universal architectures for common categories and overlook the challenges that small objects have long been thorny due to the relative deficiency of foreground points and a low tolerance for disturbances. To this end, we propose a Siamese network-based method for small object tracking in the LiDAR point cloud, which is composed of the target-awareness prototype mining (TAPM) module and the regional grid subdivision (RGS) module. The TAPM module adopts the reconstruction mechanism of the masked decoder to learn the prototype in the feature space, aiming to highlight the presence of foreground points that will facilitate the subsequent location of small objects. Through the above prototype is capable of accentuating the small object of interest, the positioning deviation in feature maps still leads to high tracking errors. To alleviate this issue, the RGS module is proposed to recover the fine-grained features of the search region based on ViT and pixel shuffle layers. In addition, apart from the normal settings, we elaborately design a scaling experiment to evaluate the robustness of the different trackers on small objects. Extensive experiments on KITTI and nuScenes demonstrate that our method can effectively improve the tracking performance of small targets without affecting normal-sized objects.

Figures (8)

• Figure 1: The function of the proposed module. The target-awareness prototype mining module successfully recovers 60 foreground points, as (a) and (b) shown. The regional grid subdivision module upsamples the bird's eye view map with a resolution of $38 \times 56$ to $76 \times 112$ one, as (c) and (d) shown.
• Figure 2: Corrosion caused by convolution. When performing the first convolution, the information in the blue area will converge to pixel $X_1$ along the direction of the blue arrow. Similarly, the information in the yellow area will converge to $X_2$. When performing the second convolution, due to the aggregation of $X_3$ information by $X_2$, $X_1$ will also indirectly receive $X_3$ information. The result of corrosion is the shrinkage of the effective area as (b) shown.
• Figure 3: The overall architecture of our method. The template point cloud and the search region point cloud are fed into a shared encoder to generate respective geometric features. Then the template features are embedded into search region features by the relation modeling module. Subsequently, the target-awareness prototype mining module will highlight the foreground features and enrich them through several self-attention layers. Finally, the target position is predicted by a bev-based detection head accompanied by a regional grid subdivision module.
• Figure 4: The architecture of the regional grid subdivision module, which consists of a Vit layer and a pixel shuffle layer. Firstly, the bird's eye view feature map decouples into patches, and the patches will be flattened and added with position embedding. They will interact and generate reconstructed features by a series of transformer encoders. Finally, each 128-channel pixel feature is divided into four 32-channel pixel features by the pixel shuffle layer.
• Figure 5: Visualization results of the pedestrian category. We selected two typical sequences in the KITTI datasets to compare the tracking results of different methods.