MemFusionMap: Working Memory Fusion for Online Vectorized HD Map Construction

TL;DR

A working memory fusion module is contributed that improves the model's memory capacity to reason across a history of frames and a novel temporal over-lap heatmap is designed to explicitly inform the model about the temporal overlap information and vehicle trajectory in the Bird's Eye View space.

Abstract

High-definition (HD) maps provide environmental information for autonomous driving systems and are essential for safe planning. While existing methods with single-frame input achieve impressive performance for online vectorized HD map construction, they still struggle with complex scenarios and occlusions. We propose MemFusionMap, a novel temporal fusion model with enhanced temporal reasoning capabilities for online HD map construction. Specifically, we contribute a working memory fusion module that improves the model's memory capacity to reason across a history of frames. We also design a novel temporal overlap heatmap to explicitly inform the model about the temporal overlap information and vehicle trajectory in the Bird's Eye View space. By integrating these two designs, MemFusionMap significantly outperforms existing methods while also maintaining a versatile design for scalability. We conduct extensive evaluation on open-source benchmarks and demonstrate a maximum improvement of 5.4% in mAP over state-of-the-art methods. The project page for MemFusionMap is https://song-jingyu.github.io/MemFusionMap

Figures (9)

• Figure 1: We propose MemFusionMap to improve the temporal reasoning capability for online vectorized HD map construction. MemFusionMap takes in multi-view images and leverages the proposed working memory fusion module and temporal overlap heatmap to output a vectorized HD map online.
• Figure 2: Overall architecture of MemFusionMap. MemFusionMap takes in multi-view images and extracts a BEV feature via an image encoder and a BEV feature encoder. The encoded BEV feature is fused with working memory features and the temporal overlap heatmap to obtain a unified BEV feature. The unified feature is fed into a decoder head to generate HD map prediction.
• Figure 3: Example propagation process of the temporal overlap heatmap from $t_0$ to $t_1$.
• Figure 4: Example temporal overlap heatmap resulted from different speeds. The comparison demonstrates that the overlap heatmap encodes vehicle moving speed.
• Figure 5: Example temporal overlap heatmap resulted from a vehicle turning. We highlight the two noticeable edges at the corner. This pattern encodes insights into the vehicle's trajectory.