Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trajectory Prediction Meets Large Language Models: A Survey

Yi Xu, Ruining Yang, Yitian Zhang, Jianglin Lu, Mingyuan Zhang, Yizhou Wang, Lili Su, Yun Fu

TL;DR

The paper addresses how large language models can enhance trajectory prediction for autonomous systems by recasting forecasting tasks through language-centric paradigms. It organizes the literature into five interconnected directions, from language-modeling-based forecasting to language-driven data generation and interpretability, and provides a critical synthesis of modeling choices, datasets, and evaluation. Key findings show that LLMs enable few-shot generalization, controllable scenario synthesis, and reasoning-enabled explanations, yet face persistent challenges in numerical grounding, spatial fidelity, and computational efficiency. The survey offers a structured roadmap with open challenges and future directions to advance language-grounded trajectory forecasting and its safe deployment.

Abstract

Recent advances in large language models (LLMs) have sparked growing interest in integrating language-driven techniques into trajectory prediction. By leveraging their semantic and reasoning capabilities, LLMs are reshaping how autonomous systems perceive, model, and predict trajectories. This survey provides a comprehensive overview of this emerging field, categorizing recent work into five directions: (1) Trajectory prediction via language modeling paradigms, (2) Direct trajectory prediction with pretrained language models, (3) Language-guided scene understanding for trajectory prediction, (4) Language-driven data generation for trajectory prediction, (5) Language-based reasoning and interpretability for trajectory prediction. For each, we analyze representative methods, highlight core design choices, and identify open challenges. This survey bridges natural language processing and trajectory prediction, offering a unified perspective on how language can enrich trajectory prediction.

Trajectory Prediction Meets Large Language Models: A Survey

TL;DR

The paper addresses how large language models can enhance trajectory prediction for autonomous systems by recasting forecasting tasks through language-centric paradigms. It organizes the literature into five interconnected directions, from language-modeling-based forecasting to language-driven data generation and interpretability, and provides a critical synthesis of modeling choices, datasets, and evaluation. Key findings show that LLMs enable few-shot generalization, controllable scenario synthesis, and reasoning-enabled explanations, yet face persistent challenges in numerical grounding, spatial fidelity, and computational efficiency. The survey offers a structured roadmap with open challenges and future directions to advance language-grounded trajectory forecasting and its safe deployment.

Abstract

Recent advances in large language models (LLMs) have sparked growing interest in integrating language-driven techniques into trajectory prediction. By leveraging their semantic and reasoning capabilities, LLMs are reshaping how autonomous systems perceive, model, and predict trajectories. This survey provides a comprehensive overview of this emerging field, categorizing recent work into five directions: (1) Trajectory prediction via language modeling paradigms, (2) Direct trajectory prediction with pretrained language models, (3) Language-guided scene understanding for trajectory prediction, (4) Language-driven data generation for trajectory prediction, (5) Language-based reasoning and interpretability for trajectory prediction. For each, we analyze representative methods, highlight core design choices, and identify open challenges. This survey bridges natural language processing and trajectory prediction, offering a unified perspective on how language can enrich trajectory prediction.

Paper Structure

This paper contains 16 sections, 3 equations, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Trajectory Prediction Formulation
  4. Key Concepts in Large Language Models
  5. Taxonomy
  6. Trajectory Prediction via Language Modeling Paradigms
  7. Direct Trajectory Prediction with Language-Based Predictors
  8. Language-Guided Scene Understanding for Trajectory Prediction
  9. Language-Driven Data Generation for Trajectory Prediction
  10. Language-Based Reasoning and Interpretability for Trajectory Prediction
  11. Challenges and Future Directions
  12. Conclusion and Discussion
  13. Background on Trajectory Prediction
  14. Survey Methodology
  15. Relation with Other Surveys
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Growth of language-based trajectory prediction papers (2022–2025) across four categories. Only methods explicitly incorporating natural language or pretrained language models (PLMs/LLMs) are counted.
  • Figure 2: Taxonomy of language-based trajectory prediction methods. We categorize representative works (2022–2025) into five groups based on the functional role of language: language modeling paradigms, direct language-based prediction, scene understanding, data generation, and reasoning & interpretability.