Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CATPlan: Loss-based Collision Prediction in End-to-End Autonomous Driving

Ziliang Xiong, Shipeng Liu, Nathaniel Helgesen, Joakim Johnander, Per-Erik Forssen

TL;DR

This work tackles the crucial safety challenge of uncertainty in planned trajectories for end-to-end autonomous driving by introducing CATPlan, a lightweight, transformer-based loss-prediction module. CATPlan decodes motion and planning embeddings from state-of-the-art AD models to estimate the collision loss and output a probabilistic collision risk, enabling proactive safety actions. It reframes collision prediction as binary classification driven by the planner's collision loss, and employs cross-attention between planning and agent-motion queries to achieve strong performance against a Gaussian Mixture Model baseline on NeuroNCAP, with notable improvements on real-world nuScenes data as well. The results show substantial gains and highlight domain gaps between sequences as a major factor limiting generalization, underscoring the importance of uncertainty quantification for safer end-to-end autonomous driving systems.

Abstract

In recent years, there has been increased interest in the design, training, and evaluation of end-to-end autonomous driving (AD) systems. One often overlooked aspect is the uncertainty of planned trajectories predicted by these systems, despite awareness of their own uncertainty being key to achieve safety and robustness. We propose to estimate this uncertainty by adapting loss prediction from the uncertainty quantification literature. To this end, we introduce a novel light-weight module, dubbed CATPlan, that is trained to decode motion and planning embeddings into estimates of the collision loss used to partially supervise end-to-end AD systems. During inference, these estimates are interpreted as collision risk. We evaluate CATPlan on the safety-critical, nerf-based, closed-loop benchmark NeuroNCAP and find that it manages to detect collisions with a $54.8\%$ relative improvement to average precision over a GMM-based baseline in which the predicted trajectory is compared to the forecasted trajectories of other road users. Our findings indicate that the addition of CATPlan can lead to safer end-to-end AD systems and hope that our work will spark increased interest in uncertainty quantification for such systems.

CATPlan: Loss-based Collision Prediction in End-to-End Autonomous Driving

TL;DR

This work tackles the crucial safety challenge of uncertainty in planned trajectories for end-to-end autonomous driving by introducing CATPlan, a lightweight, transformer-based loss-prediction module. CATPlan decodes motion and planning embeddings from state-of-the-art AD models to estimate the collision loss and output a probabilistic collision risk, enabling proactive safety actions. It reframes collision prediction as binary classification driven by the planner's collision loss, and employs cross-attention between planning and agent-motion queries to achieve strong performance against a Gaussian Mixture Model baseline on NeuroNCAP, with notable improvements on real-world nuScenes data as well. The results show substantial gains and highlight domain gaps between sequences as a major factor limiting generalization, underscoring the importance of uncertainty quantification for safer end-to-end autonomous driving systems.

Abstract

In recent years, there has been increased interest in the design, training, and evaluation of end-to-end autonomous driving (AD) systems. One often overlooked aspect is the uncertainty of planned trajectories predicted by these systems, despite awareness of their own uncertainty being key to achieve safety and robustness. We propose to estimate this uncertainty by adapting loss prediction from the uncertainty quantification literature. To this end, we introduce a novel light-weight module, dubbed CATPlan, that is trained to decode motion and planning embeddings into estimates of the collision loss used to partially supervise end-to-end AD systems. During inference, these estimates are interpreted as collision risk. We evaluate CATPlan on the safety-critical, nerf-based, closed-loop benchmark NeuroNCAP and find that it manages to detect collisions with a relative improvement to average precision over a GMM-based baseline in which the predicted trajectory is compared to the forecasted trajectories of other road users. Our findings indicate that the addition of CATPlan can lead to safer end-to-end AD systems and hope that our work will spark increased interest in uncertainty quantification for such systems.

Paper Structure

This paper contains 15 sections, 11 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Method
  4. Preliminaries: Loss Prediction
  5. Collision Prediction
  6. CATPlan Architecture
  7. Baseline: Trajectory GMM
  8. Bagging Ensemble, Mixup, and Focal Loss
  9. Experiments
  10. Safety-critical Scenarios
  11. Real-World Data
  12. Ablation
  13. Domain Gap Analysis
  14. Limitations
  15. Conclusion

Figures (3)

  • Figure 1: Basic synopsis of the CATPlan model. While conventional End-to-End Autonomous Driving Models output a plan for the vehicle to enact, our model is trained on the embeddings from such models and outputs the probability of a collision happening during that planned route. This can enable a self-driving vehicle to take emergency action or switch to driver assistance.
  • Figure 2: Qualitative Examples on NeuroNCAP Simulation Dataset with UniAD as the end-to-end planner. The left figure presents ground truth objects (gray) alongside predicted objects (color-coded by category) and their forecasted trajectories (blue-yellow gradient line) in the bird's eye view. It also depicts the ego-vehicle (black), the planned trajectory (black), and the reference trajectory (red) indicating its intended turn. The right figures are the inputs of 3 front cameras.
  • Figure 3: 2D t-SNE plots for plan queries $h_\text{plan}$ from UniAD on nuScenes and NeuroNCAP. It is better to review in color.