Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhance Planning with Physics-informed Safety Controller for End-to-end Autonomous Driving

Hang Zhou, Haichao Liu, Hongliang Lu, Dan Xu, Jun Ma, Yiding Ji

TL;DR

FusionAssurance tackles the safety gaps of learning-based end-to-end autonomous driving by coupling a transformer-based perception/planning network with a physics-informed safety controller that combines Model Predictive Control (MPC) with Potential Fields (PF). The approach uses multi-sensor fusion to predict waypoints and scene signals, and then constrains the low-level control through an MPC objective augmented by obstacle and front-vehicle PF terms, enabling safe obstacle avoidance and overtaking in unseen scenarios. Key contributions include the integration of PF-based safety into MPC within an end-to-end framework, tunable safety parameters without retraining, and demonstrated gains on the CARLA benchmark over state-of-the-art methods. The results indicate practical improvements in driving score and route completion, highlighting the method’s potential to enhance real-world safety for neural planners in dynamic traffic."

Abstract

Recent years have seen a growing research interest in applications of Deep Neural Networks (DNN) on autonomous vehicle technology. The trend started with perception and prediction a few years ago and it is gradually being applied to motion planning tasks. Despite the performance of networks improve over time, DNN planners inherit the natural drawbacks of Deep Learning. Learning-based planners have limitations in achieving perfect accuracy on the training dataset and network performance can be affected by out-of-distribution problem. In this paper, we propose FusionAssurance, a novel trajectory-based end-to-end driving fusion framework which combines physics-informed control for safety assurance. By incorporating Potential Field into Model Predictive Control, FusionAssurance is capable of navigating through scenarios that are not included in the training dataset and scenarios where neural network fail to generalize. The effectiveness of the approach is demonstrated by extensive experiments under various scenarios on the CARLA benchmark.

Enhance Planning with Physics-informed Safety Controller for End-to-end Autonomous Driving

TL;DR

FusionAssurance tackles the safety gaps of learning-based end-to-end autonomous driving by coupling a transformer-based perception/planning network with a physics-informed safety controller that combines Model Predictive Control (MPC) with Potential Fields (PF). The approach uses multi-sensor fusion to predict waypoints and scene signals, and then constrains the low-level control through an MPC objective augmented by obstacle and front-vehicle PF terms, enabling safe obstacle avoidance and overtaking in unseen scenarios. Key contributions include the integration of PF-based safety into MPC within an end-to-end framework, tunable safety parameters without retraining, and demonstrated gains on the CARLA benchmark over state-of-the-art methods. The results indicate practical improvements in driving score and route completion, highlighting the method’s potential to enhance real-world safety for neural planners in dynamic traffic."

Abstract

Recent years have seen a growing research interest in applications of Deep Neural Networks (DNN) on autonomous vehicle technology. The trend started with perception and prediction a few years ago and it is gradually being applied to motion planning tasks. Despite the performance of networks improve over time, DNN planners inherit the natural drawbacks of Deep Learning. Learning-based planners have limitations in achieving perfect accuracy on the training dataset and network performance can be affected by out-of-distribution problem. In this paper, we propose FusionAssurance, a novel trajectory-based end-to-end driving fusion framework which combines physics-informed control for safety assurance. By incorporating Potential Field into Model Predictive Control, FusionAssurance is capable of navigating through scenarios that are not included in the training dataset and scenarios where neural network fail to generalize. The effectiveness of the approach is demonstrated by extensive experiments under various scenarios on the CARLA benchmark.
Paper Structure (19 sections, 8 equations, 8 figures, 2 tables)

This paper contains 19 sections, 8 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Network Model
  5. Physics-informed Safety Controller
  6. Vehicle Dynamics
  7. Potential Field Function
  8. Model Predictive Control Formulation
  9. Neural Network and Safety Controller Integration
  10. Experiments
  11. Environment, Data Collection and Benchmark
  12. Effectiveness Analysis on Corner Cases
  13. Case 1(Fig. \ref{['fig:bad_trajectory']})
  14. Case 2(Fig. \ref{['fig:trajectory']})
  15. Case 3(Fig. \ref{['fig:mpcVSmpc_pf']})
  16. ...and 4 more sections

Figures (8)

  • Figure 1: Popular frameworks of Autonomous Driving schwarting2018planning. Green shaded is the proposed framework pipeline of FusionAssurance. Compared to other frameworks, FusionAssurance is less modular and its DNN output can be guided by safety controller.
  • Figure 2: Overall structure of FussionAssurance consists of two parts. 1) a transformer-based network that integrates the target location, multi-camera data, and lidar data to generate the predicted waypoint, BEV obstacle map, traffic light state, and junction probability for autonomous driving. 2) a safety controller that comprises Model Predictive Controller with Potential Functions. The safety controller takes the output of the neural network and sensor data of GPS, Compass and Speedometer to produce safe and optimal low-level control actions.
  • Figure 3: Test Examples of FusionAssurance completing various scenarios under different wearther conditions.
  • Figure 4: Case when unfeasible trajectory is generated by neural network.
  • Figure 5: The left figure shows the agent get into Deadlock situation by the neural network planner. The right figure shows the guided trajectory of FusionAssurance under the same planning network.
  • ...and 3 more figures