Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

VPIT: Real-time Embedded Single Object 3D Tracking Using Voxel Pseudo Images

Illia Oleksiienko, Paraskevi Nousi, Nikolaos Passalis, Anastasios Tefas, Alexandros Iosifidis

TL;DR

VPIT tackles real-time 3D single-object tracking on embedded hardware by using voxel pseudo images in BEV, enabling a 2D-like Siamese tracker to operate on 3D data. It introduces a multi-rotation search over $2K+1$ rotated BEV regions to jointly estimate the position and yaw rotation $\alpha$ while keeping object size constant in BEV. Evaluations on KITTI show that VPIT is the fastest tracker with competitive $Success$ and $Precision$, and real-time benchmarks on embedded devices reveal robust tracking with minimal frame drops despite latency constraints. By bridging 2D Siamese tracking with 3D BEV representations and emphasizing efficiency, VPIT demonstrates a practical path for deploying fast 3D SOT on resource-limited robots.

Abstract

In this paper, we propose a novel voxel-based 3D single object tracking (3D SOT) method called Voxel Pseudo Image Tracking (VPIT). VPIT is the first method that uses voxel pseudo images for 3D SOT. The input point cloud is structured by pillar-based voxelization, and the resulting pseudo image is used as an input to a 2D-like Siamese SOT method. The pseudo image is created in the Bird's-eye View (BEV) coordinates, and therefore the objects in it have constant size. Thus, only the object rotation can change in the new coordinate system and not the object scale. For this reason, we replace multi-scale search with a multi-rotation search, where differently rotated search regions are compared against a single target representation to predict both position and rotation of the object. Experiments on KITTI Tracking dataset show that VPIT is the fastest 3D SOT method and maintains competitive Success and Precision values. Application of a SOT method in a real-world scenario meets with limitations such as lower computational capabilities of embedded devices and a latency-unforgiving environment, where the method is forced to skip certain data frames if the inference speed is not high enough. We implement a real-time evaluation protocol and show that other methods lose most of their performance on embedded devices, while VPIT maintains its ability to track the object.

VPIT: Real-time Embedded Single Object 3D Tracking Using Voxel Pseudo Images

TL;DR

VPIT tackles real-time 3D single-object tracking on embedded hardware by using voxel pseudo images in BEV, enabling a 2D-like Siamese tracker to operate on 3D data. It introduces a multi-rotation search over rotated BEV regions to jointly estimate the position and yaw rotation while keeping object size constant in BEV. Evaluations on KITTI show that VPIT is the fastest tracker with competitive and , and real-time benchmarks on embedded devices reveal robust tracking with minimal frame drops despite latency constraints. By bridging 2D Siamese tracking with 3D BEV representations and emphasizing efficiency, VPIT demonstrates a practical path for deploying fast 3D SOT on resource-limited robots.

Abstract

In this paper, we propose a novel voxel-based 3D single object tracking (3D SOT) method called Voxel Pseudo Image Tracking (VPIT). VPIT is the first method that uses voxel pseudo images for 3D SOT. The input point cloud is structured by pillar-based voxelization, and the resulting pseudo image is used as an input to a 2D-like Siamese SOT method. The pseudo image is created in the Bird's-eye View (BEV) coordinates, and therefore the objects in it have constant size. Thus, only the object rotation can change in the new coordinate system and not the object scale. For this reason, we replace multi-scale search with a multi-rotation search, where differently rotated search regions are compared against a single target representation to predict both position and rotation of the object. Experiments on KITTI Tracking dataset show that VPIT is the fastest 3D SOT method and maintains competitive Success and Precision values. Application of a SOT method in a real-world scenario meets with limitations such as lower computational capabilities of embedded devices and a latency-unforgiving environment, where the method is forced to skip certain data frames if the inference speed is not high enough. We implement a real-time evaluation protocol and show that other methods lose most of their performance on embedded devices, while VPIT maintains its ability to track the object.
Paper Structure (12 sections, 18 equations, 7 figures, 5 tables)

This paper contains 12 sections, 18 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related works
  3. Proposed method
  4. Model architecture
  5. Training
  6. Inference
  7. Experiments
  8. Comparison with state-of-the-art
  9. Real-time evaluation
  10. Ablation study
  11. Limitations and future work
  12. Conclusions

Figures (7)

  • Figure 1: Structure of PointPillars 3D object detection model. The RPN is a 2D CNN that takes a pseudo image as input.
  • Figure 2: Structure of the proposed Voxel Pseudo Image Tracking model. The input point cloud is voxelized and processed with the PointPillars' Pillar Feature Network to create a voxel pseudo image, which serves as an input to the Siamese model. The Feature Generation Network (FGN), which is a convolutional subnetwork of the PointPillars' RPN, processes the target and search regions to create corresponding features that are then compared to find a position of the best similarity.
  • Figure 3: An example of a directional Gaussian penalty used with linear extrapolation, a corresponding score map and target and search pseudo images. The cyan color on the score map represents negative values.
  • Figure 4: Target features of the same object at different frames.
  • Figure 5: Merged target features of the same object at different frames with $0.5$ merge scale. Starting from frame 10, the object representation drifts to the right and completely loses the target at frame 30.
  • ...and 2 more figures