Diffusion-SAFE: Shared Autonomy Framework with Diffusion for Safe Human-to-Robot Driving Handover

TL;DR

This work tackles safe, proactive handover in shared autonomy for driving by combining two diffusion-based policies: a long-horizon evaluator that predicts human-aligned trajectories and a short-horizon copilot that generates expert trajectories. By modulating the forward and reverse diffusion processes, Diffusion-SAFE enables gradual, safe handovers and robust multimodal action sampling, reducing abrupt transitions. The system is trained on separate human and expert datasets, validated in both CarRacing-v0 simulations and ROS-based real-world race cars, and benchmarked against baselines like LSTM-GMM and BET. Results show high handover success, strong human-behavior prediction, and favorable safety and computation metrics, highlighting the practical viability of diffusion-driven shared autonomy for vehicle control.

Abstract

Safe handover in shared autonomy for vehicle control is well-established in modern vehicles. However, avoiding accidents often requires action several seconds in advance. This necessitates understanding human driver behavior and an expert control strategy for seamless intervention when a collision or unsafe state is predicted. We propose Diffusion-SAFE, a closed-loop shared autonomy framework leveraging diffusion models to: (1) predict human driving behavior for detection of potential risks, (2) generate safe expert trajectories, and (3) enable smooth handovers by blending human and expert policies over a short time horizon. Unlike prior works which use engineered score functions to rate driving performance, our approach enables both performance evaluation and optimal action sequence generation from demonstrations. By adjusting the forward and reverse processes of the diffusion-based copilot, our method ensures a gradual transition of control authority, by mimicking the drivers' behavior before intervention, which mitigates abrupt takeovers, leading to smooth transitions. We evaluated Diffusion-SAFE in both simulation (CarRacing-v0) and real-world (ROS-based race car), measuring human-driving similarity, safety, and computational efficiency. Results demonstrate a 98.5\% successful handover rate, highlighting the framework's effectiveness in progressively correcting human actions and continuously sampling optimal robot actions.

Figures (9)

• Figure 1: Closed-loop framework with two diffusion policies: an evaluator to predict human intent, and a copilot to provide optimal trajectories and ensure smooth control transitions during safety-critical situations.
• Figure 2: Overview of Diffusion-SAFE framework architecture. The evaluator model processes observations and action sequences, sampling future action sequences aligned with human intent in a simulated environment. The copilot model generates and executes expert action sequences when the human performance score falls below a predefined threshold.
• Figure 3: Noise Estimator Architecture: U-Net design with residual connections, positional embedding of step $t$, and conditioning matrix $\mathbf{C}_{t}$. Double convolution block (DC in the figure).
• Figure 4: Representative track examples in simulation and real. In total, 30 tracks were generated, 15 in the real world and 15 in simulation.
• Figure 5: Comparison on critical threshold $\gamma_0$: the blue and red lines represent the smoothness and unsafe rate of the handover process, respectively.