Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NextStop: An Improved Tracker For Panoptic LIDAR Segmentation Data

Nirit Alkalay, Roy Orfaig, Ben-Zion Bobrovsky

TL;DR

NextStop addresses weaknesses in 4D panoptic LiDAR tracking by embedding Kalman-filter motion estimation, Hungarian data association, and a tracklet-state prioritization into a two-stage tracking pipeline that converts per-frame panoptic results into consistent per-point labels. The two-stage framework first establishes robust bounding-box tracks (Stage1) and then assigns per-point IDs (Stage2) using memory-guided label mapping, overlap handling, and instance association to maintain temporal consistency. Evaluated on SemanticKITTI with the LSTQ metric, NextStop shows notable gains, particularly for small objects like pedestrians and bicyclists, reducing ID switches and enabling earlier tracking, outperforming 4D-PLS and 4D-STOP, and also improving segmentation accuracy (S_cls). The work demonstrates practical improvements for stable, long-term tracking in autonomous driving and provides open-source code for reproducibility and further research.

Abstract

4D panoptic LiDAR segmentation is essential for scene understanding in autonomous driving and robotics, combining semantic and instance segmentation with temporal consistency. Current methods, like 4D-PLS and 4D-STOP, use a tracking-by-detection methodology, employing deep learning networks to perform semantic and instance segmentation on each frame. To maintain temporal consistency, large-size instances detected in the current frame are compared and associated with instances within a temporal window that includes the current and preceding frames. However, their reliance on short-term instance detection, lack of motion estimation, and exclusion of small-sized instances lead to frequent identity switches and reduced tracking performance. We address these issues with the NextStop1 tracker, which integrates Kalman filter-based motion estimation, data association, and lifespan management, along with a tracklet state concept to improve prioritization. Evaluated using the LiDAR Segmentation and Tracking Quality (LSTQ) metric on the SemanticKITTI validation set, NextStop demonstrated enhanced tracking performance, particularly for small-sized objects like people and bicyclists, with fewer ID switches, earlier tracking initiation, and improved reliability in complex environments. The source code is available at https://github.com/AIROTAU/NextStop

NextStop: An Improved Tracker For Panoptic LIDAR Segmentation Data

TL;DR

NextStop addresses weaknesses in 4D panoptic LiDAR tracking by embedding Kalman-filter motion estimation, Hungarian data association, and a tracklet-state prioritization into a two-stage tracking pipeline that converts per-frame panoptic results into consistent per-point labels. The two-stage framework first establishes robust bounding-box tracks (Stage1) and then assigns per-point IDs (Stage2) using memory-guided label mapping, overlap handling, and instance association to maintain temporal consistency. Evaluated on SemanticKITTI with the LSTQ metric, NextStop shows notable gains, particularly for small objects like pedestrians and bicyclists, reducing ID switches and enabling earlier tracking, outperforming 4D-PLS and 4D-STOP, and also improving segmentation accuracy (S_cls). The work demonstrates practical improvements for stable, long-term tracking in autonomous driving and provides open-source code for reproducibility and further research.

Abstract

4D panoptic LiDAR segmentation is essential for scene understanding in autonomous driving and robotics, combining semantic and instance segmentation with temporal consistency. Current methods, like 4D-PLS and 4D-STOP, use a tracking-by-detection methodology, employing deep learning networks to perform semantic and instance segmentation on each frame. To maintain temporal consistency, large-size instances detected in the current frame are compared and associated with instances within a temporal window that includes the current and preceding frames. However, their reliance on short-term instance detection, lack of motion estimation, and exclusion of small-sized instances lead to frequent identity switches and reduced tracking performance. We address these issues with the NextStop1 tracker, which integrates Kalman filter-based motion estimation, data association, and lifespan management, along with a tracklet state concept to improve prioritization. Evaluated using the LiDAR Segmentation and Tracking Quality (LSTQ) metric on the SemanticKITTI validation set, NextStop demonstrated enhanced tracking performance, particularly for small-sized objects like people and bicyclists, with fewer ID switches, earlier tracking initiation, and improved reliability in complex environments. The source code is available at https://github.com/AIROTAU/NextStop
Paper Structure (26 sections, 10 equations, 11 figures, 13 tables)

This paper contains 26 sections, 10 equations, 11 figures, 13 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Motivation
  5. NextStop Tracker
  6. The Bounding-Box Tracker (Stage 1)
  7. Detection
  8. Motion-Based Prediction
  9. Tracklets State
  10. Data Association
  11. Class Majority
  12. Prioritization
  13. Bounding-box to Point Label (Stage2)
  14. Associating Tracked Bounding Boxes with Points
  15. Managing Overlapping Points
  16. ...and 11 more sections

Figures (11)

  • Figure 1: NextStop Block Diagram
  • Figure 2: Stage 1: Base Block Diagram of the Bounding Box Tracker
  • Figure 3: Stage 1: Bounding Box Tracker Complete Block Diagram
  • Figure 4: Stage 2: From Bounding Box to Points with ID
  • Figure 5: Stage 2: Bounding-box to Label Per Point Block Diagram
  • ...and 6 more figures