DiffRefiner: Coarse to Fine Trajectory Planning via Diffusion Refinement with Semantic Interaction for End to End Autonomous Driving

TL;DR

DiffRefiner tackles the multimodal nature of ego-vehicle trajectory prediction by integrating a discriminative, anchor-based Proposal Decoder with a diffusion-based Refiner in a coarse-to-fine framework. The method introduces a Fine-Grained Semantic Interaction Module to enforce scene conformity during refinement and uses a two-stage training objective that aligns perception, proposal, and refinement tasks. Empirical results on NAVSIM v2 and Bench2Drive demonstrate state-of-the-art performance, with ablations validating the benefits of the hierarchical design, FGSI, and limited denoising steps for real-time applicability. The approach advances end-to-end autonomous driving by enabling accurate, context-aware, and safety-conscious trajectory generation that respects road semantics and dynamic interactions. Collectively, DiffRefiner offers a practical, scalable pathway for high-fidelity motion planning in complex urban environments.

Abstract

Unlike discriminative approaches in autonomous driving that predict a fixed set of candidate trajectories of the ego vehicle, generative methods, such as diffusion models, learn the underlying distribution of future motion, enabling more flexible trajectory prediction. However, since these methods typically rely on denoising human-crafted trajectory anchors or random noise, there remains significant room for improvement. In this paper, we propose DiffRefiner, a novel two-stage trajectory prediction framework. The first stage uses a transformer-based Proposal Decoder to generate coarse trajectory predictions by regressing from sensor inputs using predefined trajectory anchors. The second stage applies a Diffusion Refiner that iteratively denoises and refines these initial predictions. In this way, we enhance the performance of diffusion-based planning by incorporating a discriminative trajectory proposal module, which provides strong guidance for the generative refinement process. Furthermore, we design a fine-grained denoising decoder to enhance scene compliance, enabling more accurate trajectory prediction through enhanced alignment with the surrounding environment. Experimental results demonstrate that DiffRefiner achieves state-of-the-art performance, attaining 87.4 EPDMS on NAVSIM v2, and 87.1 DS along with 71.4 SR on Bench2Drive, thereby setting new records on both public benchmarks. The effectiveness of each component is validated via ablation studies as well.

Figures (12)

• Figure 1: Comparison of different paradigms for end-to-end planning: (a) single-stage discriminative approach, (b) single-stage generative diffusion method, and (c) our proposed coarse-to-fine framework integrates discriminative proposal construction with generative diffusion refinement.
• Figure 2: Overview of the proposed DiffRefiner. The DiffRefiner architecture comprises three primary components: a BEV encoder, an perception module, and a planning module, which sequentially perform scene representation learning, perception, and motion planning. The planning module is further decomposed into two submodules: (a) a proposal decoder, which employs a discriminative approach to produce coarse proposals that capture the overall motion trend; and (b) a diffusion refiner, which refines the proposals by leveraging a fine-grained denoising decoder conditioned on explicitly modeled scene semantics, thereby generating a final trajectory that better complies with environmental constraints.
• Figure 3: Illustration of the detailed architecture of the refiner and the Fine-Grained Semantic Interaction Module (FGSIM).
• Figure 4: Visualization of representative examples of DiffusionDrive liao2025diffusiondrive and our method. (a) and (b) illustrate cases in which our method achieves better collision avoidance compared with DiffusionDrive, whereas (c) and (d) demonstrate cases where our method exhibits improved compliance with lane constraints.
• Figure 5: Qualitative examples of the proposal refinement process on NAVSIM. Red regions indicate roads, blue represents centerlines, pink denotes vehicles, and green corresponds to walkways. The ego vehicle is positioned at the center of the bottom area. Gray lines depict all 20 predicted candidate trajectories.