Recover to Predict: Progressive Retrospective Learning for Variable-Length Trajectory Prediction

TL;DR

A Progressive Retrospective Framework is proposed, which gradually aligns features from incomplete observations with those from complete ones via a cascade of retrospective units via a cascade of retrospective units, and is plug-and-play with existing methods.

Abstract

Trajectory prediction is critical for autonomous driving, enabling safe and efficient planning in dense, dynamic traffic. Most existing methods optimize prediction accuracy under fixed-length observations. However, real-world driving often yields variable-length, incomplete observations, posing a challenge to these methods. A common strategy is to directly map features from incomplete observations to those from complete ones. This one-shot mapping, however, struggles to learn accurate representations for short trajectories due to significant information gaps. To address this issue, we propose a Progressive Retrospective Framework (PRF), which gradually aligns features from incomplete observations with those from complete ones via a cascade of retrospective units. Each unit consists of a Retrospective Distillation Module (RDM) and a Retrospective Prediction Module (RPM), where RDM distills features and RPM recovers previous timesteps using the distilled features. Moreover, we propose a Rolling-Start Training Strategy (RSTS) that enhances data efficiency during PRF training. PRF is plug-and-play with existing methods. Extensive experiments on datasets Argoverse 2 and Argoverse 1 demonstrate the effectiveness of PRF. Code is available at https://github.com/zhouhao94/PRF.

Figures (9)

• Figure 1: Fig. \ref{['fig:teaser-a']} and Fig. \ref{['fig:teaser-b']} display two common scenarios that yield variable-length, incomplete trajectories. Fig. \ref{['fig:teaser-c']} and Fig. \ref{['fig:teaser-d']} respectively present the mADE$_6$ and mFDE$_6$ results for the original DeMo zhang2024decoupling, DeMo with Isolated Training (DeMo-IT), and DeMo with PRF (DeMo-PRF) under varying observation lengths.
• Figure 2: Overview of the PRF. A cascade of retrospective units progressively distills features of varying-length inputs, aligning them with those from complete ones to improve prediction performance. Each unit includes a Retrospective Distillation Module (RDM) that distills features to longer observations and a Retrospective Prediction Module (RPM) that recovers omitted history from the distilled features.
• Figure 3: Illustration of the (a) RDM and (b) RPM. RDM employs a residual-based distillation strategy, featuring a logit branch that generates a gating vector and a residual branch that learns from the omitted history. RPM employs a decoupled query strategy, utilizing mode queries for multimodal trajectory proposals and state queries for trajectory refinement, with the proposals serving as anchors.
• Figure 4: Qualitative results on the Argoverse 2 validation set. Incomplete observations, predicted trajectories, and ground truth trajectories are shown in yellow, green, and pink, respectively. Our predictions align more closely with the ground truth than other methods.
• Figure 5: t-SNE visualization of (a) direct and (b) progressive distillation strategies. Features distilled from 10 to 50 are shown in yellow, while features of the standard length 50 are shown in blue.