LHPF: Look back the History and Plan for the Future in Autonomous Driving

TL;DR

LHPF is introduced, an imitation learning planner that integrates historical planning information that not only surpasses existing advanced learning-based planners in planning performance but also marks the first instance of a purely learning-based planner outperforming the expert.

Abstract

Decision-making and planning in autonomous driving critically reflect the safety of the system, making effective planning imperative. Current imitation learning-based planning algorithms often merge historical trajectories with present observations to predict future candidate paths. However, these algorithms typically assess the current and historical plans independently, leading to discontinuities in driving intentions and an accumulation of errors with each step in a discontinuous plan. To tackle this challenge, this paper introduces LHPF, an imitation learning planner that integrates historical planning information. Our approach employs a historical intention aggregation module that pools historical planning intentions, which are then combined with a spatial query vector to decode the final planning trajectory. Furthermore, we incorporate a comfort auxiliary task to enhance the human-like quality of the driving behavior. Extensive experiments using both real-world and synthetic data demonstrate that LHPF not only surpasses existing advanced learning-based planners in planning performance but also marks the first instance of a purely learning-based planner outperforming the expert. Additionally, the application of the historical intention aggregation module across various backbones highlights the considerable potential of the proposed method. The code will be made publicly available.

Figures (7)

• Figure 1: Comparison of imitation learning planners with and without historical intentions. The ego vehicle, shown as the green rectangle, initiates an overtaking maneuver at $t_0$. Both planners in (a) and (b) exhibit high open-loop performance at $t_0$; however, due to the absence of historical intentions and the accumulation of errors, method (a) diverges significantly from the ground truth by $t_2$. In contrast, the planner in (b) incorporates historical planning embeddings, preserving the initial driving intention accurately.
• Figure 2: Planning with Historical Intentions. A stack of planners generates historical planning embeddings at each time step, which are stored in a historical intention pool and combined with reference line queries. Spatio-temporal queries are then aggregated and processed using self and cross attention with scene context as keys and values. Finally, the current planning embedding is passed through a multi-layer perceptron to generate future trajectories and scores.
• Figure 3: Planning Stream Demonstration. The diagram illustrates planning across consecutive frames, with dashed lines highlighting significant overlap that reveals the continuity and evolution of driving behaviors.
• Figure 4: Demonstration of Reactive Closed-Loop Simulation in Five Representative Scenarios. The rectangles represent the vehicles, with the orange rectangle denoting the autonomous ego vehicle. Additional legends are provided in (a).
• Figure 5: Planning performance comparison with different historical planning intervals (upper) and training epochs (lower).