Motion Forecasting for Autonomous Vehicles: A Survey

TL;DR

The surveyed work tackles the problem of forecasting vehicle and agent motions for autonomous driving by delineating two primary pipelines: scenario-based and perception-based forecasting. It formalizes the problem, surveys datasets and evaluation metrics, and categorizes methods into supervised and self-supervised learning, detailing encoding and decoding architectures across temporal-spatial, visual, and transformer-based approaches. Key contributions include a comprehensive taxonomy of inputs, prediction types, and model architectures, along with critical challenges such as HDMaps fusion, inter-agent dynamics, and multimodality. The paper also highlights emerging directions, including self-supervised pretraining, language-model-inspired sequence modeling, and diffusion models for controllable multi-agent trajectory prediction, pointing to significant practical impact for robust, scalable AV systems.

Abstract

In recent years, the field of autonomous driving has attracted increasingly significant public interest. Accurately forecasting the future behavior of various traffic participants is essential for the decision-making of Autonomous Vehicles (AVs). In this paper, we focus on both scenario-based and perception-based motion forecasting for AVs. We propose a formal problem formulation for motion forecasting and summarize the main challenges confronting this area of research. We also detail representative datasets and evaluation metrics pertinent to this field. Furthermore, this study classifies recent research into two main categories: supervised learning and self-supervised learning, reflecting the evolving paradigms in both scenario-based and perception-based motion forecasting. In the context of supervised learning, we thoroughly examine and analyze each key element of the methodology. For self-supervised learning, we summarize commonly adopted techniques. The paper concludes and discusses potential research directions, aiming to propel progress in this vital area of AV technology.

Figures (7)

• Figure 1: The increasing trend within the research community is evidenced by the growing number of articles on Google Scholar that include the keywords "Autonomous Driving" and "Vehicle Motion Forecasting" published between 2019 and 2024.
• Figure 2: Overview of motion forecasting methodologies, challenges, and future directions for autonomous vehicle trajectory prediction.
• Figure 3: Different types of traffic participants (agents).
• Figure 4: The relationship between marginal prediction and joint multi-agent prediction. The main difference between them is that the former has a single $TA$; The latter has multiple $TAs$.
• Figure 5: Comparison of scenario-based and perception-based motion forecasting pipelines in autonomous driving.