SegLocNet: Multimodal Localization Network for Autonomous Driving via Bird's-Eye-View Segmentation

TL;DR

SegLocNet tackles GNSS-denied urban localization by leveraging BEV semantic segmentation trained on multimodal inputs (multi-view images and LiDAR) and matching the resulting BEV map to a lightweight, unified prior map. The method avoids regression-based pose estimation and uses an exhaustive BEV–map matching strategy with a 3-DoF pose formulation $p=(x,y,\theta)$, enabling robust localization across both HD and SD maps. A unified binary map mask enables seamless use of OpenStreetMap-based SD maps alongside HD maps, while a dedicated BEV encoder–fusion–decoder backbone produces BEV segmentation used for precise pose solving. Experiments on nuScenes and Argoverse show state-of-the-art performance and strong generalization, including cross-city transfers, with ablations demonstrating the importance of map elements and road-width settings. The approach offers strong interpretability, efficient integration into perception stacks, and public release of code and models.

Abstract

Robust and accurate localization is critical for autonomous driving. Traditional GNSS-based localization methods suffer from signal occlusion and multipath effects in urban environments. Meanwhile, methods relying on high-definition (HD) maps are constrained by the high costs associated with the construction and maintenance of HD maps. Standard-definition (SD) maps-based methods, on the other hand, often exhibit unsatisfactory performance or poor generalization ability due to overfitting. To address these challenges, we propose SegLocNet, a multimodal GNSS-free localization network that achieves precise localization using bird's-eye-view (BEV) semantic segmentation. SegLocNet employs a BEV segmentation network to generate semantic maps from multiple sensor inputs, followed by an exhaustive matching process to estimate the vehicle's ego pose. This approach avoids the limitations of regression-based pose estimation and maintains high interpretability and generalization. By introducing a unified map representation, our method can be applied to both HD and SD maps without any modifications to the network architecture, thereby balancing localization accuracy and area coverage. Extensive experiments on the nuScenes and Argoverse datasets demonstrate that our method outperforms the current state-of-the-art methods, and that our method can accurately estimate the ego pose in urban environments without relying on GNSS, while maintaining strong generalization ability. Our code and pre-trained model will be released publicly.

Figures (4)

• Figure 1: To address the localization challenges due to signal occlusion and multipath errors in urban environments, we propose SegLocNet, a novel multimodal localization network that leverages multi-view images and the LiDAR point cloud to construct the local BEV semantic map of the surroundings. The precise pose estimation is achieved through the alignment of the BEV semantic map with the lightweight prior map.
• Figure 2: The overall architecture of SegLocNet. We use a camera BEV encoder and a LiDAR BEV encoder to extract BEV features in a low-dimensional embedded space from the respective sensor inputs. Subsequently, the multimodal BEV features are fused with a lightweight convolutional neural network. The fused BEV features are then upsampled to the target resolution by the segmentation decoder to produce the BEV segmentation. Meanwhile, for each observation, a tile of local map is queried from the prior maps based on the given initial position. The vehicle's ego pose is estimated by exhaustively matching the BEV segmentation with the local prior map.
• Figure 3: Visualization of different map processing methods.
• Figure 4: Visualization of localization results on the Argoverse dataset. The red and blue arrows represent the GT pose and the predicted pose respectively.