LiDAR-based Real-Time Object Detection and Tracking in Dynamic Environments

TL;DR

This paper tackles real-time dynamic object detection and tracking in dynamic environments using LiDAR data alone. It proposes a mapless pipeline that converts LiDAR Point Clouds into intensity images, applies a Gaussian-based low-frequency feature extraction, and uses intensity-based ego-motion estimation with region growing to reconstruct moving objects. The approach achieves high detection accuracy and recall, demonstrates resilience to front-end odometry drift, and runs in real time, outperforming state-of-the-art methods on a dataset collected with a Spot robot. The results suggest significant practical impact for autonomous navigation in cluttered, dynamic settings and offer a path toward robust SLAM integration.

Abstract

In dynamic environments, the ability to detect and track moving objects in real-time is crucial for autonomous robots to navigate safely and effectively. Traditional methods for dynamic object detection rely on high accuracy odometry and maps to detect and track moving objects. However, these methods are not suitable for long-term operation in dynamic environments where the surrounding environment is constantly changing. In order to solve this problem, we propose a novel system for detecting and tracking dynamic objects in real-time using only LiDAR data. By emphasizing the extraction of low-frequency components from LiDAR data as feature points for foreground objects, our method significantly reduces the time required for object clustering and movement analysis. Additionally, we have developed a tracking approach that employs intensity-based ego-motion estimation along with a sliding window technique to assess object movements. This enables the precise identification of moving objects and enhances the system's resilience to odometry drift. Our experiments show that this system can detect and track dynamic objects in real-time with an average detection accuracy of 88.7\% and a recall rate of 89.1\%. Furthermore, our system demonstrates resilience against the prolonged drift typically associated with front-end only LiDAR odometry. All of the source code, labeled dataset, and the annotation tool are available at: https://github.com/MISTLab/lidar_dynamic_objects_detection.git

Figures (11)

• Figure 1: Illustration of dynamic object detection. Top: The raw intensity image. Middle: Dynamic objects are highlighted in red within the intensity image. Bottom: Dynamic points corresponding to these objects are extracted from the raw 3D point cloud, showing their spatial distribution.
• Figure 2: The framework of our dynamic object detection system, consisting of three main components: data preprocessing, object tracking, and dynamic object detection. The data preprocessing component converts unordered LiDAR point cloud data into intensity images and extracts low-frequency components to represent foreground objects. Object tracking involves ego-motion estimation, clustering, and cluster association. The ego-motion estimation component calculates the robot's movement between frames using intensity-based front-end odometry, providing real-time movement data despite potential noise and drift. The dynamic object detection component leverages data association results to track objects and identify seed points of dynamic objects, followed by a region growing algorithm to reconstruct the complete point set of dynamic objects from these seed points. This system processes LiDAR data in real-time to detect dynamic objects in complex environments, using the low-frequency components to represent foreground objects and enable real-time tracking across multiple frames.
• Figure 3: Dynamic Object Tracking: to adapt to the drift of the front-end odometry, we accumulate the transformation matrix between the current frame and the first frame within a time window. This approach allows us to ignore historical drift and only account for the drift within the current time window. By transforming all frames into the same coordinate system, we can calculate the distance between the centroids of clusters in consecutive frames and match corresponding clusters between frames. This method enables real-time tracking of all detected foreground objects. By analyzing the movement of clusters within the sliding window, we can effectively detect dynamic objects. In the picture, three frames within the time window are transformed into the initial frame's coordinate system. Although all clusters appear to move due to the odometry drift, the movement of static objects is minimal and usually follows a zero-mean Gaussian distribution. Therefore, we can filter out dynamic objects by accumulating the distance between the cluster's centroid in the current frame and the initial frame.
• Figure 4: Spot robot navigating a dynamic environment. Equipped with an OS0-64 LiDAR sensor, the Spot robot moves through a large area filled with pedestrians as moving objects. Both the pedestrians and the Spot robot exhibit random movements, creating a constantly changing environment that challenges the robot's navigation and object detection capabilities.
• Figure 5: Point cloud map of the environment with dynamic objects. The dynamic points create blurring in the map as they move, highlighting the challenges of accurately mapping in dynamic environments.