A Knowledge-Driven Diffusion Policy for End-to-End Autonomous Driving Based on Expert Routing

TL;DR

End-to-end autonomous driving faces challenges of generalization, multi-modality, and long-horizon planning. The paper presents a knowledge-driven diffusion policy (KDP) that integrates diffusion-based generative modeling with a sparse mixture-of-experts router to produce temporally coherent action sequences and compose reusable driving knowledge. Key contributions include modeling driving knowledge as abstract experts, a Top-K routing mechanism with load balancing and mutual information regularization, and a Transformer-backed diffusion backbone that achieves superior performance in ramp, intersection, and roundabout scenarios. Results show higher success rates, lower collision risk, and smoother control compared to baselines, with ablations validating the necessity of sparse routing and temporal modeling for cross-scenario generalization and knowledge reuse.

Abstract

End-to-end autonomous driving remains constrained by the difficulty of producing adaptive, robust, and interpretable decision-making across diverse scenarios. Existing methods often collapse diverse driving behaviors, lack long-horizon consistency, or require task-specific engineering that limits generalization. This paper presents KDP, a knowledge-driven diffusion policy that integrates generative diffusion modeling with a sparse mixture-of-experts routing mechanism. The diffusion component generates temporally coherent action sequences, while the expert routing mechanism activates specialized and reusable experts according to context, enabling modular knowledge composition. Extensive experiments across representative driving scenarios demonstrate that KDP achieves consistently higher success rates, reduced collision risk, and smoother control compared to prevailing paradigms. Ablation studies highlight the effectiveness of sparse expert activation and the Transformer backbone, and activation analyses reveal structured specialization and cross-scenario reuse of experts. These results establish diffusion with expert routing as a scalable and interpretable paradigm for knowledge-driven end-to-end autonomous driving.

Figures (11)

• Figure 1: End-to-end autonomous driving paradigms: (a) policy networks, (b) language-based model, and (c) diffusion policies, where single-modality refers to policies that collapse multiple feasible maneuvers into a single trajectory, whereas multi-modality denotes the ability to generate diverse valid trajectories under the same driving context.
• Figure 2: Framework of the proposed Knowledge-Driven Diffusion Policy. Scene inputs condition a diffusion-based policy to generate multi-modal, temporally coherent action sequences. A Mixture-of-Experts module refines these sequences by activating experts interpreted as abstract knowledge units, whose combinations express diverse and extensible driving skills such as interaction, maneuvering, and adaptation.
• Figure 3: Diffusion-based action generation, where expert demonstrations are perturbed by forward diffusion and recovered by reverse denoising to produce executable action sequences for simulation.
• Figure 4: MoE-based knowledge routing framework, where a Top-$K$ router dynamically selects specialized experts to compose modular driving knowledge for adaptive policy learning.
• Figure 5: Three selected representative and challenging scenarios , including (a) In Ramp, (b) Intersection, and (c) Roundabout, which increase in difficulty and test different aspects of end-to-end autonomous driving capabilities.