MetaDAT: Generalizable Trajectory Prediction via Meta Pre-training and Data-Adaptive Test-Time Updating

TL;DR

A meta-learning framework is proposed to directly optimize the predictor for fast and accurate online adaptation, which performs bi-level optimization on the performance of simulated test-time adaptation tasks during pre-training and at test time, and a data-adaptive model updating mechanism that dynamically adjusts the predefined learning rates and updating frequencies based on online partial derivatives and hard sample selection.

Abstract

Existing trajectory prediction methods exhibit significant performance degradation under distribution shifts during test time. Although test-time training techniques have been explored to enable adaptation, current approaches rely on an offline pre-trained predictor that lacks online learning flexibility. Moreover, they depend on fixed online model updating rules that do not accommodate the specific characteristics of test data. To address these limitations, we first propose a meta-learning framework to directly optimize the predictor for fast and accurate online adaptation, which performs bi-level optimization on the performance of simulated test-time adaptation tasks during pre-training. Furthermore, at test time, we introduce a data-adaptive model updating mechanism that dynamically adjusts the predefined learning rates and updating frequencies based on online partial derivatives and hard sample selection. This mechanism enables the online learning rate to suit the test data, and focuses on informative hard samples to enhance efficiency. Experiments are conducted on various challenging cross-dataset distribution shift scenarios, including nuScenes, Lyft, and Waymo. Results demonstrate that our method achieves superior adaptation accuracy, surpassing state-of-the-art test-time training methods for trajectory prediction. Additionally, our method excels under suboptimal learning rates and high FPS demands, showcasing its robustness and practicality.

Figures (5)

• Figure 1: (a) Illustration for the test-time training and online evaluation protocol for trajectory prediction. (b) Previous offline pre-training overlooks the offline-online misalignment between pre-training and adaptation. In contrast, our method aligns these two phases via meta pre-training (MP). Additionally, instead of using fixed updating rules, we propose data-adaptive test-time updating by dynamic optimization of the learning rate $\alpha$ (DLO) and additional hard-sample-driven updates (HSD).
• Figure 2: Overview of our MetaDAT framework. MetaDAT first performs meta pre-training across the simulated test-time training tasks on the source dataset, then the pre-trained model goes through data-adaptive test-time training on the target dataset under distribution shifts. Newly proposed modules in our framework are highlighted in orange, while the TTT baseline is in purple.
• Figure 3: Qualitative visualizations on test-time training results. The top row shows the prediction results without adaptation, while the bottom row indicates the adaptation results using test-time training. We only visualize the best modality for multi-modal predictions.
• Figure 4: Qualitative visualizations on multi-modal prediction results for TTT methods. Left: T4P. Right: MetaDAT.
• Figure 5: Prediction accuracy and execution FPS on nuS→ Lyft long-term experiments. The number next to each data point indicates the updating frequency. We plot results for frequencies of 1,2,3,5. MetaDAT demonstrates better prediction performance at the same FPS.