Perfect Prediction or Plenty of Proposals? What Matters Most in Planning for Autonomous Driving

TL;DR

This work questions the assumed benefit of perfect predictive foresight in Integrated Prediction and Planning (IPP) for autonomous driving. By evaluating DTPP, GameFormer, and DIPP under learned, perfect, and no-prediction conditions, the authors show that even ground-truth future behaviors offer limited planning advantages, highlighting a bottleneck in how predictions influence planning. They identify a critical need for high-quality proposal generation and introduce SPDM, a spline-based extension of PDM that generates diverse, realistic ego trajectories. SPDM significantly improves performance on highly interactive and out-of-distribution scenarios (notably interPlan), suggesting that future IPP progress hinges more on proposal generation than on prediction accuracy alone. The findings argue for rethinking IPP design to prioritize robust, diverse proposals and efficient collision-aware checks at test time.

Abstract

Traditionally, prediction and planning in autonomous driving (AD) have been treated as separate, sequential modules. Recently, there has been a growing shift towards tighter integration of these components, known as Integrated Prediction and Planning (IPP), with the aim of enabling more informed and adaptive decision-making. However, it remains unclear to what extent this integration actually improves planning performance. In this work, we investigate the role of prediction in IPP approaches, drawing on the widely adopted Val14 benchmark, which encompasses more common driving scenarios with relatively low interaction complexity, and the interPlan benchmark, which includes highly interactive and out-of-distribution driving situations. Our analysis reveals that even access to perfect future predictions does not lead to better planning outcomes, indicating that current IPP methods often fail to fully exploit future behavior information. Instead, we focus on high-quality proposal generation, while using predictions primarily for collision checks. We find that many imitation learning-based planners struggle to generate realistic and plausible proposals, performing worse than PDM - a simple lane-following approach. Motivated by this observation, we build on PDM with an enhanced proposal generation method, shifting the emphasis towards producing diverse but realistic and high-quality proposals. This proposal-centric approach significantly outperforms existing methods, especially in out-of-distribution and highly interactive settings, where it sets new state-of-the-art results.

Figures (5)

• Figure 1: Visualization of a real-world scenario from nuPlan Val14 dataset in reactive simulation. The ego vehicle attempts to cross an intersection while a pedestrian is crossing. Using the state-of-the-art DTPP huang2024dtpp planner with (a) perfect predictions (i.e., the actual responses of agents under the given plan) results in a collision, whereas DTPP with (b) learned predictions successfully avoids the pedestrian. Even when provided with ground-truth future motion, a state-of-the-art planner generates a colliding trajectory. In contrast, it avoids collisions when relying on its own predictions, although these predictions are noisy and inaccurate.
• Figure 2: Visualization of $N_p = 15$ ego trajectory proposals generated by various approaches. Each proposal’s endpoint is indicated by a red dot. Subfigures (a)–(d) present proposals from learning-based methods, while subfigure (e) illustrates proposals from our rule-based method. In this highly interactive scenario, the ego vehicle must perform multiple consecutive lane changes to reach the designated goal lane, highlighted in green. Among the approaches presented, only our proposed method considers a lane change in this scenario.
• Figure 3: Comparative analysis of subscores between learned and perfect predictions when utilized within DTPP, GameFormer, GameFormer + DIPP, and DIPP across the Val14 and interPlan benchmark in reactive simulation. Markers in the upper triangle denote better performance with learned predictions, whereas markers in the lower triangle denote better performance with perfect predictions. Subscore ‘Lane Changes to Goal’ is exclusively available in the interPlan benchmark.
• Figure 4: Subscore performance comparison between learned-based proposal generators and the rule-based PDM-Closed on the interPlan benchmark in reactive simulation. Each point represents a specific subscore, with the y-axis indicating PDM-Closed performance and the x-axis showing performance of the corresponding learned-based generator. For the learned-based methods, the best proposal is selected using the PDM scoring function.
• Figure 5: Figure (a) shows performance of PDM across varying numbers of proposals on the Val14 and interPlan benchmarks. NR/R: non-reactive/reactive simulation. Figure (b) shows runtime analysis across varying numbers of proposals, averaged over 10 distinct scenarios.