FlexMap: Generalized HD Map Construction from Flexible Camera Configurations

TL;DR

FlexMap tackles HD map construction under heterogeneous, uncalibrated camera configurations by removing reliance on explicit camera poses and 2D-to-BEV projections. It introduces a geometry Transformer backbone with spatial-temporal enhancement and a camera-aware decoder that uses latent camera tokens to adapt attention across views, producing per-view vectorized map elements. The approach achieves strong pose-free performance across 1–6 camera configurations on nuScenes and generalizes to Argoverse 2, demonstrating robustness to missing views and sensor variations. This enables scalable, crowdsourced HD mapping for real-world autonomous driving deployments, reducing calibration requirements and improving deployment practicality.

Abstract

High-definition (HD) maps provide essential semantic information of road structures for autonomous driving systems, yet current HD map construction methods require calibrated multi-camera setups and either implicit or explicit 2D-to-BEV transformations, making them fragile when sensors fail or camera configurations vary across vehicle fleets. We introduce FlexMap, unlike prior methods that are fixed to a specific N-camera rig, our approach adapts to variable camera configurations without any architectural changes or per-configuration retraining. Our key innovation eliminates explicit geometric projections by using a geometry-aware foundation model with cross-frame attention to implicitly encode 3D scene understanding in feature space. FlexMap features two core components: a spatial-temporal enhancement module that separates cross-view spatial reasoning from temporal dynamics, and a camera-aware decoder with latent camera tokens, enabling view-adaptive attention without the need for projection matrices. Experiments demonstrate that FlexMap outperforms existing methods across multiple configurations while maintaining robustness to missing views and sensor variations, enabling more practical real-world deployment.

Figures (10)

• Figure 1: An illustration of HD map construction from flexible camera configurations.
• Figure 2: We visualize the BEV feature map used by a BEV-based decoder (e.g., MapTR maptr2023), and a query (green) whose attention for a target location concentrates near a view boundary (red box). While the full six-camera setting (6PV) yields coherent vectors, using only the front camera (1PV) causes fragmentation around the boundary due to missing-view contributions in the BEV features. The VGGT result shows that geometry foundation models can produce geometry-grounded representations even under a sequence of front camera views, motivating a flexible and calibration-free HD map construction pipeline.
• Figure 3: Overview of our method. Multi-camera temporal sequences are processed by a geometry transformer to extract multi-scale features, followed by a spatial-temporal attention module for temporal aggregation. A camera-aware decoder with hierarchical queries then predicts vectorized maps per view. As illustrated, when using the front and back-left cameras as inputs, the model outputs a vectorized map for each provided view.
• Figure 4: Camera-specific map element statistics for nuScenes caesar_nuscenes_2020 dataset after FOV clipping. (a) Average number of elements, (b) average polyline length, (c) orientation distribution (longitudinal vs. lateral). Front and back cameras (F, B) observe denser, longer structures, while side cameras (FL, FR, BL, BR) capture fewer, shorter, and largely lateral elements. (d) shows that vectorized map elements projected in perspective view differ across cameras. The front view primarily contains longitudinal lane structures aligned with the driving direction, whereas the back view exhibits distinct geometry due to the reversed viewing direction.
• Figure 5: Qualitative comparison on nuScenes dataset. We visualize predictions from different methods across challenging scenarios using a single front camera as input. FlexMap, our pose-free method, produces geometrically consistent predictions (third column) that match ground truth (second column), outperforming the baseline methods with knowledge of ground truth camera pose.