TS-CGNet: Temporal-Spatial Fusion Meets Centerline-Guided Diffusion for BEV Mapping

TL;DR

TS-CGNet tackles BEV semantic mapping under occlusions and adverse conditions by fusing temporal-prior information with centerline-guided diffusion. It comprises a Local Mapping System, a Temporal-Spatial Aligner Module (TSAM), and a Centerline-Guided Diffusion Model (CGDM), integrating historical maps and OpenStreetMap centerlines as conditioning cues. The approach yields consistent improvements across BEV HD and semantic mapping tasks on nuScenes and robustness benchmarks, including $1.9\%$, $1.73\%$, and $2.87\%$ gains at various ranges, and an average $2.92\%$ improvement under weather and sensor disturbances. These results demonstrate the value of combining prior knowledge with diffusion-based reconstruction for reliable autonomous driving perception, with code to be released publicly.

Abstract

Bird's Eye View (BEV) perception technology is crucial for autonomous driving, as it generates top-down 2D maps for environment perception, navigation, and decision-making. Nevertheless, the majority of current BEV map generation studies focusing on visual map generation lack depth-aware reasoning capabilities. They exhibit limited efficacy in managing occlusions and handling complex environments, with a notable decline in perceptual performance under adverse weather conditions or low-light scenarios. Therefore, this paper proposes TS-CGNet, which leverages Temporal-Spatial fusion with Centerline-Guided diffusion. This visual framework, grounded in prior knowledge, is designed for integration into any existing network for building BEV maps. Specifically, this framework is decoupled into three parts: Local mapping system involves the initial generation of semantic maps using purely visual information; The Temporal-Spatial Aligner Module (TSAM) integrates historical information into mapping generation by applying transformation matrices; The Centerline-Guided Diffusion Model (CGDM) is a prediction module based on the diffusion model. CGDM incorporates centerline information through spatial-attention mechanisms to enhance semantic segmentation reconstruction. We construct BEV semantic segmentation maps by our methods on the public nuScenes and the robustness benchmarks under various corruptions. Our method improves 1.90%, 1.73%, and 2.87% for perceived ranges of 60x30m, 120x60m, and 240x60m in the task of BEV HD mapping. TS-CGNet attains an improvement of 1.92% for perceived ranges of 100x100m in the task of BEV semantic mapping. Moreover, TS-CGNet achieves an average improvement of 2.92% in detection accuracy under varying weather conditions and sensor interferences in the perception range of 240x60m. The source code will be publicly available at https://github.com/krabs-H/TS-CGNet.

Figures (4)

• Figure 1: Overview of TS-CGNet, the proposed framework for BEV semantic mapping using historical and centerline priors. In addition, the bar chart illustrates the robustness under varying environments and sensor influences between ours and the state-of-the-art method jiang2024p.
• Figure 2: The structure of TS-CGNet. It consists of a local mapping system, the temporal-spatial aligner module, and the centerline-guided diffusion model. Integrating historical maps can provide additional prior knowledge, allowing the diffusion model to fit a more comprehensive map distribution.
• Figure 3: The framework of CGDM. It integrates ${F}_{f}$ as the prompt and $Map_o$ as the condition into the UNet decoder to refine noisy features into accurate BEV maps.
• Figure 4: The visualization of the BEV HD mapping task using TS-CGNet. The semantic colors indicate that 'blue' denotes (Div.), 'green' denotes (Ped.), and 'purple' denotes (Bound.).