Compressed Map Priors for 3D Perception

TL;DR

CMP introduces a differentiable, end-to-end learnable compressed map prior that encodes persistent spatial context via a multi-resolution hash embedding and binarized features. It fuses this prior with existing multi-view 3D detectors through lightweight fusion mechanisms (dense-conCAT/convolution or cross-attention), achieving consistent improvements across BEVDet, BEVFormer, and PETR on nuScenes with only ~3% additional runtime and a memory footprint of about $32~\text{KB}/\text{km}^2$. Key findings include substantial gains for BEV-based architectures, strong resilience to partial sensor input, and clear memory-accuracy trade-offs that favor moderate embedding sizes (optimal around $T=2^{16}$). The approach emphasizes end-to-end optimization of priors, differentiable propagation of gradients into the map representation, and persistent knowledge transfer across traversals, suggesting practical benefits for real-world autonomous driving deployments in repetitive environments.

Abstract

Human drivers rarely travel where no person has gone before. After all, thousands of drivers use busy city roads every day, and only one can claim to be the first. The same holds for autonomous computer vision systems. The vast majority of the deployment area of an autonomous vision system will have been visited before. Yet, most autonomous vehicle vision systems act as if they are encountering each location for the first time. In this work, we present Compressed Map Priors (CMP), a simple but effective framework to learn spatial priors from historic traversals. The map priors use a binarized hashmap that requires only $32\text{KB}/\text{km}^2$, a $20\times$ reduction compared to the dense storage. Compressed Map Priors easily integrate into leading 3D perception systems at little to no extra computational costs, and lead to a significant and consistent improvement in 3D object detection on the nuScenes dataset across several architectures.

Figures (9)

• Figure 1: Multiple traversals of a scene in the nuScenes dataset caesar2020nuscenes between the train and val split.
• Figure 2: Comparison of how vision systems incorporate spatial priors. (a) Traditional approaches yang2018hdnet retrieve map annotations and fuse them with sensor features for downstream tasks. (b) Our proposed Compressed Map Priors (CMP) retrieves prior features by computing lookups from a highly compressed learnable embedding, represented by binarized features. CMP is fully differentiable and allows end-to-end learning with gradient updates through the entire pipeline.
• Figure 3: Overview. Illustration of our map representation. (1) We define a regular grid of query points in ego-vehicle coordinates. (2) For the four corners of cell this point falls in, we perform a spatial lookup in a binarized embedding. (3) Bilinear interpolation of the corner features produces a level-specific feature - we repeat this across multiple spatial resolutions and concatenate the features to produce a multi-scale feature representation. (4) Finally, we use an MLP to project the multi-scale feature into a single prior feature $x_{prior}$. We perform this in parallel for all query points to obtain the full prior $\mathbf{X}_{prior}$.
• Figure 4: Performance with different numbers of training traversals. Both methods improve as traversals increase, but CMP significantly outperforms the baseline, with gains magnifying at higher traversal counts (0, 1-3, 4-6, 7-17 traversals). Our model gracefully degrades to baseline performance in areas with no prior traversals.
• Figure 5: Performance over different distances. Average precision for the "car" class across three different distance thresholds: close (0-10 meters), medium (10-25 meters), and far (25-50 meters).