Night-Rider: Nocturnal Vision-aided Localization in Streetlight Maps Using Invariant Extended Kalman Filtering

TL;DR

This work proposes a nocturnal vision-aided localization system in streetlight maps with a novel data association and matching scheme using object detection methods and leverages the Invariant Extended Kalman Filter to fuse IMU, odometer, and camera measurements for consistent state estimation at night.

Abstract

Vision-aided localization for low-cost mobile robots in diverse environments has attracted widespread attention recently. Although many current systems are applicable in daytime environments, nocturnal visual localization is still an open problem owing to the lack of stable visual information. An insight from most nocturnal scenes is that the static and bright streetlights are reliable visual information for localization. Hence we propose a nocturnal vision-aided localization system in streetlight maps with a novel data association and matching scheme using object detection methods. We leverage the Invariant Extended Kalman Filter (InEKF) to fuse IMU, odometer, and camera measurements for consistent state estimation at night. Furthermore, a tracking recovery module is also designed for tracking failures. Experimental results indicate that our proposed system achieves accurate and robust localization with less than $0.2\%$ relative error of trajectory length in four nocturnal environments.

Figures (7)

• Figure 1: The proposed nocturnal localization system in the nighttime campus streetlight map. Top-left image shows the robot configuration. Top-right image is the trajectory (red) overlaid with the daytime satellite map for visualization. The middle images display the matches of 2D streetlight detections and 3D streetlight clusters. The bottom image depicts the estimated trajectory in the streetlight map.
• Figure 2: System overview of the nocturnal vision-aided localization system. The InEKF is utilized for state estimation.
• Figure 3: An example of match extension. The module determines more matches according to the positional relationship between the projected locations of streetlight clusters and the boxes detected by the binary segmentation method.
• Figure 4: Degeneration case. The left image shows that all positions on the blue circle satisfy the identical camera observation constraints. An inaccurate prior pose leads to deviation from the true pose. The right images show that the streetlight matches are difficult to search after the degeneration cases.
• Figure 5: Trajectories of ground truth (gray dashed line), VINS-Odom (green), and Night-Rider (blue) in four real scenes.