Physics-informed Diffusion Mamba Transformer for Real-world Driving

TL;DR

This work addresses the challenge of planning under uncertainty in autonomous driving by marrying diffusion-based trajectory modeling with physical feasibility. It introduces the Diffusion Mamba Transformer (DiMT) backbone, which incorporates a Mamba state-space module, attention, and a Mixture-of-Experts, guided by a Port-Hamiltonian Neural Network (PHNN) to enforce energy-based constraints and dynamic consistency. The method operates in continuous time and supports efficient sampling via DPM-Solver++ while maintaining safety through physics-informed corrections and hard anchoring to observed states. Empirical results on the nuPlan benchmark show significant improvements in predictive accuracy, physical plausibility, and robustness over state-of-the-art baselines, demonstrating the practical potential for safer, real-time motion planning in complex urban environments.

Abstract

Autonomous driving systems demand trajectory planners that not only model the inherent uncertainty of future motions but also respect complex temporal dependencies and underlying physical laws. While diffusion-based generative models excel at capturing multi-modal distributions, they often fail to incorporate long-term sequential contexts and domain-specific physical priors. In this work, we bridge these gaps with two key innovations. First, we introduce a Diffusion Mamba Transformer architecture that embeds mamba and attention into the diffusion process, enabling more effective aggregation of sequential input contexts from sensor streams and past motion histories. Second, we design a Port-Hamiltonian Neural Network module that seamlessly integrates energy-based physical constraints into the diffusion model, thereby enhancing trajectory predictions with both consistency and interpretability. Extensive evaluations on standard autonomous driving benchmarks demonstrate that our unified framework significantly outperforms state-of-the-art baselines in predictive accuracy, physical plausibility, and robustness, thereby advancing safe and reliable motion planning.

Figures (5)

• Figure 1: Pipelines for Diffusion model with refinement. (a) Adapts the traditional rules to filter infeasible trajectories. (b) Uses a two stage framework with a separately learned rule-informed network for trajectory refinement. (c) Our approach has a unified architecture which improves diffusion process with a physics-informed network.
• Figure 2: Pi-DiMT follows an encoder–fusion–diffusion design. Agent, static-object, and lane features are encoded into compact embeddings and fused via a multi-head FusionEncoder to form scene context. A DiT-based diffusion decoder, conditioned on context, route features, and the diffusion timestep, denoises trajectory tokens with the current state constrained. Finally, a Port-Hamiltonian Network provides physics-informed guidance via end-to-end training.
• Figure 3: Port-Hamiltonian Network Guidance. The energy exchange is calculated by the dominant kinetic energy which is caused by acceleration. The estimated acceleration is learned by the MLP with input of weight average acceleration $a_{wavg}(t-1)$ of past 0.5s and current scenario embedding.
• Figure 4: Test Examples of Pi-DiMT completing a variety of scenarios.
• Figure 5: Control analysis for diffusion models with and without Port-Hamiltonian Network in left-turn cross road scenario. The PHNN‑enhanced diffusion model (blue) demonstrates substantial improvements in reducing acceleration and steering jerk.