Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Where Do You Go? Pedestrian Trajectory Prediction using Scene Features

Mohammad Ali Rezaei, Fardin Ayar, Ehsan Javanmardi, Manabu Tsukada, Mahdi Javanmardi

TL;DR

Pedestrian trajectory prediction must account for both social interactions and environmental context. The authors introduce ScenePTP, which fuses Sparse Graph Convolutional Network–based pedestrian interactions with scene features obtained via Real-ESRGAN enhancement and OneFormer semantic maps, using a cross-attention mechanism before a Temporal Convolutional Network predictor. The approach achieves state-of-the-art results on ETH/UCY, with an average $\mathrm{ADE} = 0.252$ m and $\mathrm{FDE} = 0.372$ m, and shows notable gains in the HOTEL scene ($\mathrm{ADE}=0.145$ m). Ablation studies confirm the value of semantic maps and image restoration for richer environmental context and more accurate forecasting, highlighting the practical impact for autonomous systems navigating crowded urban spaces.

Abstract

Accurate prediction of pedestrian trajectories is crucial for enhancing the safety of autonomous vehicles and reducing traffic fatalities involving pedestrians. While numerous studies have focused on modeling interactions among pedestrians to forecast their movements, the influence of environmental factors and scene-object placements has been comparatively underexplored. In this paper, we present a novel trajectory prediction model that integrates both pedestrian interactions and environmental context to improve prediction accuracy. Our approach captures spatial and temporal interactions among pedestrians within a sparse graph framework. To account for pedestrian-scene interactions, we employ advanced image enhancement and semantic segmentation techniques to extract detailed scene features. These scene and interaction features are then fused through a cross-attention mechanism, enabling the model to prioritize relevant environmental factors that influence pedestrian movements. Finally, a temporal convolutional network processes the fused features to predict future pedestrian trajectories. Experimental results demonstrate that our method significantly outperforms existing state-of-the-art approaches, achieving ADE and FDE values of 0.252 and 0.372 meters, respectively, underscoring the importance of incorporating both social interactions and environmental context in pedestrian trajectory prediction.

Where Do You Go? Pedestrian Trajectory Prediction using Scene Features

TL;DR

Pedestrian trajectory prediction must account for both social interactions and environmental context. The authors introduce ScenePTP, which fuses Sparse Graph Convolutional Network–based pedestrian interactions with scene features obtained via Real-ESRGAN enhancement and OneFormer semantic maps, using a cross-attention mechanism before a Temporal Convolutional Network predictor. The approach achieves state-of-the-art results on ETH/UCY, with an average m and m, and shows notable gains in the HOTEL scene ( m). Ablation studies confirm the value of semantic maps and image restoration for richer environmental context and more accurate forecasting, highlighting the practical impact for autonomous systems navigating crowded urban spaces.

Abstract

Accurate prediction of pedestrian trajectories is crucial for enhancing the safety of autonomous vehicles and reducing traffic fatalities involving pedestrians. While numerous studies have focused on modeling interactions among pedestrians to forecast their movements, the influence of environmental factors and scene-object placements has been comparatively underexplored. In this paper, we present a novel trajectory prediction model that integrates both pedestrian interactions and environmental context to improve prediction accuracy. Our approach captures spatial and temporal interactions among pedestrians within a sparse graph framework. To account for pedestrian-scene interactions, we employ advanced image enhancement and semantic segmentation techniques to extract detailed scene features. These scene and interaction features are then fused through a cross-attention mechanism, enabling the model to prioritize relevant environmental factors that influence pedestrian movements. Finally, a temporal convolutional network processes the fused features to predict future pedestrian trajectories. Experimental results demonstrate that our method significantly outperforms existing state-of-the-art approaches, achieving ADE and FDE values of 0.252 and 0.372 meters, respectively, underscoring the importance of incorporating both social interactions and environmental context in pedestrian trajectory prediction.
Paper Structure (21 sections, 10 equations, 4 figures, 2 tables)

This paper contains 21 sections, 10 equations, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Factors Affecting Pedestrian Trajectory Prediction
  3. Spatial Position of Pedestrians in the Scene
  4. Interaction with the Surrounding Environment
  5. Pedestrian Movement Preferences and Intentions
  6. Related Work
  7. Proposed Method
  8. Problem Definition
  9. Overall Model
  10. Feature Extraction Module
  11. Interaction Module
  12. Cross-Attention Module
  13. Trajectory Prediction and Loss Function
  14. Experiments
  15. Evaluation Datasets
  16. ...and 6 more sections

Figures (4)

  • Figure 1: Our model predicts future trajectories by integrating scene and graph features. To achieve this, the approach employs a cross-attention module followed by a Temporal Convolutional Network.
  • Figure 2: Architecture of our proposed Feature Extraction Module and the pedestrian trajectory prediction pipeline. The feature vectors from both modules are combined using a cross-attention mechanism.
  • Figure 3: Semantic Segmentation Comparison on ETH, HOTEL, and ZARA Datasets. OneFormer with Image restoration, delivers the most accurate and detailed segmentations, effectively distinguishing complex features better than PSPNet zhao2017pyramid and SegNet badrinarayanan2017segnet.
  • Figure 4: Effect of Image Restoration on Frame Quality and Semantic Segmentation. (a) Original low-resolution frame, (b) Restored high-resolution frame using Real-ESRGAN, (c) Semantic segmentation on the original low-resolution frame, and (d) Improved semantic segmentation on the restored frame. Image restoration enhances visual clarity, leading to more accurate and detailed segmentation results.