An End-to-End Learning-Based Multi-Sensor Fusion for Autonomous Vehicle Localization

TL;DR

This paper proposes a learning-based method that encodes sensor information using higher-order neural network features, thereby eliminating the need for uncertainty estimation and outperforms existing multi-sensor fusion methods in terms of both accuracy and robustness.

Abstract

Multi-sensor fusion is essential for autonomous vehicle localization, as it is capable of integrating data from various sources for enhanced accuracy and reliability. The accuracy of the integrated location and orientation depends on the precision of the uncertainty modeling. Traditional methods of uncertainty modeling typically assume a Gaussian distribution and involve manual heuristic parameter tuning. However, these methods struggle to scale effectively and address long-tail scenarios. To address these challenges, we propose a learning-based method that encodes sensor information using higher-order neural network features, thereby eliminating the need for uncertainty estimation. This method significantly eliminates the need for parameter fine-tuning by developing an end-to-end neural network that is specifically designed for multi-sensor fusion. In our experiments, we demonstrate the effectiveness of our approach in real-world autonomous driving scenarios. Results show that the proposed method outperforms existing multi-sensor fusion methods in terms of both accuracy and robustness. A video of the results can be viewed at https://youtu.be/q4iuobMbjME.

Figures (8)

• Figure 1: An overview of our end-to-end network for multi-sensor fusion. Our experiments demonstrate superior performance in scenarios with unreliable measurements (e.g., GNSS signal blockage), as shown on the right side of the figure.
• Figure 2: The proposed architecture processes and fuse sensor data to output vehicle poses with neural network.
• Figure 3: The relationship between predicted states and updated states.
• Figure 4: Our experimental vehicle is equipped with a $128$-line LiDAR, a dual-antenna GNSS receiver, and an IMU.
• Figure 5: Stacked column charts of error distributions: (a) Position Error and (b) Heading Error.