IRL-DAL: Safe and Adaptive Trajectory Planning for Autonomous Driving via Energy-Guided Diffusion Models

TL;DR

IRL-DAL addresses the challenge of safe, robust autonomous driving in dynamic environments by unifying imitation learning, inverse reinforcement learning, diffusion-based safety planning, and adaptive perception. The framework combines a two-phase training curriculum (imitation pre-training followed by IRL-PPO fine-tuning), a diffusion-based on-demand safety supervisor, and a Learnable Adaptive Mask (LAM) to achieve expert-like behavior with improved safety. Key innovations include FSM-aware experience replay, an energy-guided diffusion planner for runtime safety, and SAEC to provide safe corrections without terminating episodes prematurely. In Webots, IRL-DAL achieves a 96% success rate and reduces collisions to 0.05 per 1k steps, with ablations showing substantial gains from FSM replay, the diffusion planner, and LAM+SAEC.

Abstract

This paper proposes a novel inverse reinforcement learning framework using a diffusion-based adaptive lookahead planner (IRL-DAL) for autonomous vehicles. Training begins with imitation from an expert finite state machine (FSM) controller to provide a stable initialization. Environment terms are combined with an IRL discriminator signal to align with expert goals. Reinforcement learning (RL) is then performed with a hybrid reward that combines diffuse environmental feedback and targeted IRL rewards. A conditional diffusion model, which acts as a safety supervisor, plans safe paths. It stays in its lane, avoids obstacles, and moves smoothly. Then, a learnable adaptive mask (LAM) improves perception. It shifts visual attention based on vehicle speed and nearby hazards. After FSM-based imitation, the policy is fine-tuned with Proximal Policy Optimization (PPO). Training is run in the Webots simulator with a two-stage curriculum. A 96\% success rate is reached, and collisions are reduced to 0.05 per 1k steps, marking a new benchmark for safe navigation. By applying the proposed approach, the agent not only drives in lane but also handles unsafe conditions at an expert level, increasing robustness.We make our code publicly available.

Figures (7)

• Figure 1: Overview of the IRL-DAL architecture. The training process unfolds in two phases: the policy is first initialized by Behavioral Cloning (BC) and then fine-tuned with Proximal Policy Optimization (PPO) using a hybrid reward $r_{\text{total}}$. A LAM enhances state-aware perception. During RL rollouts, the Diffusion-based Adaptive Lookahead (DAL) planner serves as a safety supervisor, correcting unsafe PPO actions through energy-guided sampling so that only safe experiences are stored in the replay buffer $\mathcal{D}_{\text{PPO}}$.
• Figure 2: Architecture of the LAM and its integration with the PPO policy. Normalized speed $v_t^{\text{norm}}$ and hazard level $h_t$ modulate a vertical gradient mask via learnable parameters $\alpha_{\text{speed}}$ and $\alpha_{\text{lidar}}$. The resulting mask is concatenated with the RGB image to form a 4-channel input, enabling context-aware visual processing. LAM is trained end-to-end via BC gradients, allowing the agent to discover adaptive attention patterns that prioritize safety-critical regions.
• Figure 3: The FSM expert policy in action. It switches smoothly between the modes (Lane Following, Obstacle Avoidance, Driving Straight, Returning) using sensor-based transition rules. The FSM-aware experience replay stores each sample in its matching state buffer, which gives balanced exposure to both normal and risky driving situations.
• Figure 4: The safety pipeline in action. DAL is activated by high-risk signals, generates energy-guided safe trajectories, and blends the first action with the PPO output. The corrected experiences are stored in the replay buffer with markers, The diffusion planner acts as an active shield, allowing the agent to safely collect more high-quality expert data even in challenging scenarios.
• Figure 5: Impact of Hybrid Reward Shaping on Trajectory Smoothness. The blue trajectory illustrates the policy trained solely on the rule-based environment reward ($r_{\text{env}}$), resulting in oscillatory behavior and excessive lateral deviation. In contrast, the green trajectory incorporates the dense IRL signal ($r_{\text{IRL}}$), effectively regularizing the policy towards the expert’s kinematic profile. This results in smoother lane-change maneuvers (left) and tighter collision avoidance (right), correcting the overshooting tendencies observed in the baseline.