GTransPDM: A Graph-embedded Transformer with Positional Decoupling for Pedestrian Crossing Intention Prediction

TL;DR

This work tackles pedestrian crossing intention prediction (PCIP) under ego-vehicle motion and depth uncertainty by introducing GTransPDM, a Graph-embedded Transformer with a Position Decoupling Module (PDM). The approach fuses multi-modal cues—pedestrian position, skeleton pose, and ego-vehicle motion—through dedicated encoders and a residual graph convolutional network that feeds a Transformer for temporal reasoning, with a depth proxy derived from bounding-box area ratios. The key contributions include decoupling ego-vehicle and pedestrian lateral motion via PDM, using a depth cue to avoid costly depth estimation, and incorporating learnable edge importance in GCNs for improved spatio-temporal pose modeling; the method achieves 1–2 second-ahead predictions with state-of-the-art accuracy (PIE ≈ $0.92$, JAAD ≈ $0.87$) and real-time inference (~$0.05$ ms). This yields a lightweight, robust PCIP solution suitable for real-world autonomous driving, offering better resilience to image distortions and depth estimation challenges while maintaining practical deployment speed.

Abstract

Understanding and predicting pedestrian crossing behavioral intention is crucial for the driving safety of autonomous vehicles. Nonetheless, challenges emerge when using promising images or environmental context masks to extract various factors for time-series network modeling, causing pre-processing errors or a loss of efficiency. Typically, pedestrian positions captured by onboard cameras are often distorted and do not accurately reflect their actual movements. To address these issues, GTransPDM -- a Graph-embedded Transformer with a Position Decoupling Module -- was developed for pedestrian crossing intention prediction by leveraging multi-modal features. First, a positional decoupling module was proposed to decompose pedestrian lateral motion and encode depth cues in the image view. Then, a graph-embedded Transformer was designed to capture the spatio-temporal dynamics of human pose skeletons, integrating essential factors such as position, skeleton, and ego-vehicle motion. Experimental results indicate that the proposed method achieves 92% accuracy on the PIE dataset and 87% accuracy on the JAAD dataset, with a processing speed of 0.05ms. It outperforms the state-of-the-art in comparison.

Figures (6)

• Figure 1: Problem Definition of Pedestrian Crossing Intention Prediction (PCIP): Given $T$ frames of observations, PCIP predicts whether a pedestrian will cross the road within the next 1-2 seconds, allowing the ego-vehicle to react in advance. In this work, we developed a multi-modal fusion approach, incorporating factors such as pedestrian position, human skeleton pose, and ego-vehicle motion.
• Figure 2: Overview of the proposed framework.
• Figure 3: Visualizations of predicted pedestrian crossing intentions in challenging scenarios. Displayed are the crossing probabilities with and without our proposed PDM, where values above 50% indicate crossing. Our method performs robustly under complex conditions, including irregular intersections (a) and curved roads (e), while also capturing long-range pedestrian behavior (b), and ambiguous roadside behavior (c) and (d). Failure cases typically falling into the parallel crossing behavior during vehicle turning, keypoints outliers, and the pedestrians yielding behavior to the ego-vehicle, as shown on the right.
• Figure 4: Performance evaluations on the PIE (left) and JAAD (right) datasets across various TTE settings in both validation and test sets.
• Figure 5: Impact of observation length on model performance across PIE (left) and JAAD (right) datasets.