Real-Time Truly-Coupled Lidar-Inertial Motion Correction and Spatiotemporal Dynamic Object Detection

TL;DR

This paper addresses motion distortion in rolling lidar scans by introducing a real-time, initialization-free lidar–IMU coupling that relies on continuous IMU preintegration to parameterize an $11$-DoF trajectory. The method undistorts lidar data through a non-linear least-squares optimization using point-to-plane and point-to-line residuals, while simultaneously computing spatiotemporal normals to detect dynamic objects without global map registration. Key contributions include the 11-DoF continuous-time formulation, a feature-guided data association within a sliding window, and a learning-free dynamic object detection approach that leverages undistorted data. The approach demonstrates competitive motion-correction accuracy against state-of-the-art methods and offers robust dynamic detection suitable for real-time perception in dynamic environments, with open-source release for replication and further integration into SLAM pipelines.

Abstract

Over the past decade, lidars have become a cornerstone of robotics state estimation and perception thanks to their ability to provide accurate geometric information about their surroundings in the form of 3D scans. Unfortunately, most of nowadays lidars do not take snapshots of the environment but sweep the environment over a period of time (typically around 100 ms). Such a rolling-shutter-like mechanism introduces motion distortion into the collected lidar scan, thus hindering downstream perception applications. In this paper, we present a novel method for motion distortion correction of lidar data by tightly coupling lidar with Inertial Measurement Unit (IMU) data. The motivation of this work is a map-free dynamic object detection based on lidar. The proposed lidar data undistortion method relies on continuous preintegrated of IMU measurements that allow parameterising the sensors' continuous 6-DoF trajectory using solely eleven discrete state variables (biases, initial velocity, and gravity direction). The undistortion consists of feature-based distance minimisation of point-to-line and point-to-plane residuals in a non-linear least-square formulation. Given undistorted geometric data over a short temporal window, the proposed pipeline computes the spatiotemporal normal vector of each of the lidar points. The temporal component of the normals is a proxy for the corresponding point's velocity, therefore allowing for learning-free dynamic object classification without the need for registration in a global reference frame. We demonstrate the soundness of the proposed method and its different components using public datasets and compare them with state-of-the-art lidar-inertial state estimation and dynamic object detection algorithms.

Figures (9)

• Figure 1: The proposed method introduces a real-time IMU-lidar motion distortion algorithm that undistorts lidar data (b) to later perform dynamic object detection (c). The colour in (a) and (b) depicts the points' timestamps, while to corresponds the classification output in (c).
• Figure 2: Overview of the proposed lidar-inertial undistortion pipeline. The sensors' trajectory is modelled with continuous preintegrated measurements legentil2023latent resulting in an estimated state of only 11-DoF.
• Figure 3: Illustration of the proposed lidar feature detection as the point-to-line distance with the "neighbour line".
• Figure 4: Intuition for the spatiotemporal normal computation as a proxy for dynamic point detection, 1D in (a) and 2D in (b). The spatiotemporal normal vectors are related to the velocity of an object; for static objects, the temporal component of the normals is null (the normals have been omitted in (b) for the sake of readability) falque2023dynamic.
• Figure 5: Illustration of the proposed pipeline's timeline in terms of lidar data used for motion correction and dynamic object detection. As per our implementation, 3 segments (of $150\,\mathrm{ms}$) per temporal window are used with a sliding window increment of 1 segment. For the dynamic detection $n$, the 3 segments of the motion correction $n$ are used. In each motion correction, the lidar features from the first and last segments in the temporal window are used for data association.