Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tractable Joint Prediction and Planning over Discrete Behavior Modes for Urban Driving

Adam Villaflor, Brian Yang, Huangyuan Su, Katerina Fragkiadaki, John Dolan, Jeff Schneider

TL;DR

The paper tackles the challenge of integrating multimodal trajectory forecasting with planning in interaction-rich urban driving. It leverages anchor embeddings to parameterize discrete latent modes and performs fully reactive closed-loop planning over these modes, without retraining the forecasting model, enabling tractable autoregressive rollouts. Key contributions include a scalable, mode-space planning approach that scales linearly with horizon and agents, and strong empirical results showing proactive behavior and state-of-the-art performance on challenging CARLA benchmarks, including dynamic merging and Longest6 at realistic speeds. The work advances practical autonomous driving by enabling proactive, interaction-aware decisions in dense traffic, reducing the frozen robot problem and improving safety and efficiency in real-world scenarios.

Abstract

Significant progress has been made in training multimodal trajectory forecasting models for autonomous driving. However, effectively integrating these models with downstream planners and model-based control approaches is still an open problem. Although these models have conventionally been evaluated for open-loop prediction, we show that they can be used to parameterize autoregressive closed-loop models without retraining. We consider recent trajectory prediction approaches which leverage learned anchor embeddings to predict multiple trajectories, finding that these anchor embeddings can parameterize discrete and distinct modes representing high-level driving behaviors. We propose to perform fully reactive closed-loop planning over these discrete latent modes, allowing us to tractably model the causal interactions between agents at each step. We validate our approach on a suite of more dynamic merging scenarios, finding that our approach avoids the $\textit{frozen robot problem}$ which is pervasive in conventional planners. Our approach also outperforms the previous state-of-the-art in CARLA on challenging dense traffic scenarios when evaluated at realistic speeds.

Tractable Joint Prediction and Planning over Discrete Behavior Modes for Urban Driving

TL;DR

The paper tackles the challenge of integrating multimodal trajectory forecasting with planning in interaction-rich urban driving. It leverages anchor embeddings to parameterize discrete latent modes and performs fully reactive closed-loop planning over these modes, without retraining the forecasting model, enabling tractable autoregressive rollouts. Key contributions include a scalable, mode-space planning approach that scales linearly with horizon and agents, and strong empirical results showing proactive behavior and state-of-the-art performance on challenging CARLA benchmarks, including dynamic merging and Longest6 at realistic speeds. The work advances practical autonomous driving by enabling proactive, interaction-aware decisions in dense traffic, reducing the frozen robot problem and improving safety and efficiency in real-world scenarios.

Abstract

Significant progress has been made in training multimodal trajectory forecasting models for autonomous driving. However, effectively integrating these models with downstream planners and model-based control approaches is still an open problem. Although these models have conventionally been evaluated for open-loop prediction, we show that they can be used to parameterize autoregressive closed-loop models without retraining. We consider recent trajectory prediction approaches which leverage learned anchor embeddings to predict multiple trajectories, finding that these anchor embeddings can parameterize discrete and distinct modes representing high-level driving behaviors. We propose to perform fully reactive closed-loop planning over these discrete latent modes, allowing us to tractably model the causal interactions between agents at each step. We validate our approach on a suite of more dynamic merging scenarios, finding that our approach avoids the which is pervasive in conventional planners. Our approach also outperforms the previous state-of-the-art in CARLA on challenging dense traffic scenarios when evaluated at realistic speeds.
Paper Structure (23 sections, 5 equations, 3 figures, 2 tables)

This paper contains 23 sections, 5 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Trajectory forecasting models for driving
  4. Planning over learned forecasting models
  5. Learning to drive in CARLA
  6. Trajectory Prediction
  7. Model formulation
  8. Network architecture
  9. Training objectives
  10. Closed-Loop Planning
  11. Planning with autoregressive rollouts
  12. Evaluating ego-modes
  13. Reward function
  14. Results
  15. Model training and hyperparameters
  16. ...and 8 more sections

Figures (3)

  • Figure 1: Forecasting model architecture.
  • Figure 2: Merge scenario visualization. The red car is the ego-vehicle, the orange arrow indicates the desired merging behavior, and the blue arrows indicate the flow of traffic in the target lane.
  • Figure 3: Qualitative example of proactive merging with closed-loop planning. The top row is from our proposed closed-loop planner and the bottom row is from the open-loop variant of our planner. Frames are in sequential order from left to right. The closed-loop planner merges proactively in front of other cars, causing the car behind to yield to the ego-agent. The open-loop planner does cannot predict that the ego will affect the behavior of other cars, so it does not attempt to merge.