MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillation

TL;DR

MapDistill introduces a novel teacher–student knowledge-distillation framework to boost efficient camera-based HD map construction by transferring knowledge from a camera–LiDAR fusion teacher to a lightweight camera-only student. It combines a Dual BEV Transform module with three distillation streams—cross-modal relation distillation, dual-level BEV feature distillation, and map head distillation—to bridge modality gaps and improve HD map predictions without increasing inference cost. The approach is validated on nuScenes, showing a significant gain (over 7.7 mAP) and substantial speedups over prior methods, with extensive ablations confirming the contribution of each component. This work offers a practical, high-performance pathway for camera-only HD map construction in autonomous driving with potential for real-world deployment.

Abstract

Online high-definition (HD) map construction is an important and challenging task in autonomous driving. Recently, there has been a growing interest in cost-effective multi-view camera-based methods without relying on other sensors like LiDAR. However, these methods suffer from a lack of explicit depth information, necessitating the use of large models to achieve satisfactory performance. To address this, we employ the Knowledge Distillation (KD) idea for efficient HD map construction for the first time and introduce a novel KD-based approach called MapDistill to transfer knowledge from a high-performance camera-LiDAR fusion model to a lightweight camera-only model. Specifically, we adopt the teacher-student architecture, i.e., a camera-LiDAR fusion model as the teacher and a lightweight camera model as the student, and devise a dual BEV transform module to facilitate cross-modal knowledge distillation while maintaining cost-effective camera-only deployment. Additionally, we present a comprehensive distillation scheme encompassing cross-modal relation distillation, dual-level feature distillation, and map head distillation. This approach alleviates knowledge transfer challenges between modalities, enabling the student model to learn improved feature representations for HD map construction. Experimental results on the challenging nuScenes dataset demonstrate the effectiveness of MapDistill, surpassing existing competitors by over 7.7 mAP or 4.5X speedup.

Figures (3)

• Figure 1: Comparison of different methods on the nuScenes val dataset. We benchmark the inference speed on a single NVIDIA RTX 3090 GPU. Our method can achieve a better trade-off in both speed (FPS) and accuracy (mAP).
• Figure 2: The overview of our proposed MapDistill. It consists of a fusion-based teacher model (top) and a lightweight camera-based student model (bottom). In addition, three distillation losses are employed to enable the teacher model to transfer knowledge to the student, i.e., by instructing the student model to produce similar features and predictions, which are cross-modal relation distillation ($\mathcal{L}_{relation}$), dual-level feature distillation ($\mathcal{L}_{feature}$), and map head distillation ($\mathcal{L}_{head}$). Note that only the student model is needed for inference.
• Figure 3: Qualitative results on nuScenes val set. (a) Six camera inputs. (b) Ground-truth vectorized HD map. (c) Result of the camera-LiDAR-based teacher model. (d) Result of the camera-based student model without MapDistill (Baseline). (e) Result of the camera-based student model with MapDistill. MapDistill helps correct substantial errors in the Baseline's predictions and improves its accuracy.