Trends in Motion Prediction Toward Deployable and Generalizable Autonomy: A Revisit and Perspectives

TL;DR

The paper identifies a critical gap between motion-prediction research and real-world deployment, proposing a generalization lifecycle with lifelong data learning, robust training, deployment-time safety, and online adaptation. It delivers a thorough taxonomy of representations and modeling approaches, analyzes deployment challenges across perception-prediction-planning stacks, and surveys strategies for generalization including self-supervised learning, domain generalization/adaptation, continual learning, OOD handling, data synthesis, and foundation-model scaling. The work emphasizes evaluation from closed-loop, perception-aware, and planning-aware perspectives, arguing for tasks that better reflect system-level safety and performance. By outlining open challenges and practical perspectives, the paper aims to recalibrate the community toward progress that translates into real-world robustness and safety in autonomous systems.

Abstract

Motion prediction, recently popularized as world models, refers to the anticipation of future agent states or scene evolution, which is rooted in human cognition, bridging perception and decision-making. It enables intelligent systems, such as robots and self-driving cars, to act safely in dynamic, human-involved environments, and informs broader time-series reasoning challenges. With advances in methods, representations, and datasets, the field has seen rapid progress, reflected in quickly evolving benchmark results. Yet, when state-of-the-art methods are deployed in the real world, they often struggle to generalize to open-world conditions and fall short of deployment standards. This reveals a gap between research benchmarks, which are often idealized or ill-posed, and real-world complexity. To address this gap, this survey revisits the generalization and deployability of motion prediction models, with an emphasis on applications of robotics, autonomous driving, and human motion. We first offer a comprehensive taxonomy of motion prediction methods, covering representations, modeling strategies, application domains, and evaluation protocols. We then study two key challenges: (1) how to push motion prediction models to be deployable to realistic deployment standards, where motion prediction does not act in a vacuum, but functions as one module of closed-loop autonomy stacks - it takes input localization and perception, and informs downstream planning and control. 2) How to generalize motion prediction models from limited seen scenarios/datasets to the open-world settings. Throughout the paper, we highlight critical open challenges to guide future work, aiming to recalibrate the community's efforts, fostering progress that is not only measurable but also meaningful for real-world applications. The project webpage can be found here https://trends-in-motion-prediction-2025.github.io/.

Figures (41)

• Figure 1: Roadmap toward deployable and generalizable models for autonomous systems. (a) Models intended for real-world deployment should be developed and evaluated under realistic settings, which require: (1) adopting inter-module representations that are informative, efficient, and scalable with data; (2) managing uncertainty throughout the autonomy stack; (3) enabling joint learning across modules to promote information sharing and resolve incompatibilities; and (4) aligning evaluation with the performance of the full closed-loop system. (b) The resulting models and evaluation protocols are then integrated into a generalization cycle that: (1) absorbs diverse data to broaden the support of the training distribution; (2) learns representations capable of generalizing across a wide range of operational domains; (3) handles sporadic distribution shifts during deployment to ensure safety; and (4) adapts online to anticipate and respond to episodic distribution changes. The lifelong learning system incrementally updates the database with newly encountered OOD data to expand the data coverage, thereby initiating the next generalization cycle to further expand the operational envelope of the system.
• Figure 2: Distribution of literature discussed in this survey across key research topics and publication years, highlighting the evolution and interconnection of deployable motion prediction and generalizable motion prediction research from before 2020 to after 2024.
• Figure 3: Overview of the content covered in Chapter 2: Taxonomy of Motion Prediction Methods.
• Figure 4: Overview of the content covered in Chapter 3: Deploying Motion Prediction with Real-world Autonomous Systems.
• Figure 5: Overview of the content covered in Section 4: Generalizable Motion Prediction.