BP-SGCN: Behavioral Pseudo-Label Informed Sparse Graph Convolution Network for Pedestrian and Heterogeneous Trajectory Prediction

TL;DR

This work tackles pedestrian and heterogeneous trajectory prediction by introducing behavioral pseudo-labels learned solely from motion features, enabling label-free modeling of inter- and intra-class behaviors. BP-SGCN combines a deep unsupervised clustering module (VRNN+DEC) that yields pseudo-labels with a goal-guided sparse graph convolutional predictor, linked end-to-end via a Gumbel-Softmax estimator within a cascaded training scheme. The approach achieves state-of-the-art performance on multiple benchmarks (SDD, Argoverse 1, ETH/UCY) without relying on manual agent-class labels or scene features, and demonstrates robust improvements in both heterogeneous and pedestrian-only settings. The method's ability to capture nuanced behavioral patterns and integrate them into a predictive graph backbone has practical implications for autonomous driving and surveillance, reducing labeling costs while enhancing prediction accuracy and reliability.

Abstract

Trajectory prediction allows better decision-making in applications of autonomous vehicles or surveillance by predicting the short-term future movement of traffic agents. It is classified into pedestrian or heterogeneous trajectory prediction. The former exploits the relatively consistent behavior of pedestrians, but is limited in real-world scenarios with heterogeneous traffic agents such as cyclists and vehicles. The latter typically relies on extra class label information to distinguish the heterogeneous agents, but such labels are costly to annotate and cannot be generalized to represent different behaviors within the same class of agents. In this work, we introduce the behavioral pseudo-labels that effectively capture the behavior distributions of pedestrians and heterogeneous agents solely based on their motion features, significantly improving the accuracy of trajectory prediction. To implement the framework, we propose the Behavioral Pseudo-Label Informed Sparse Graph Convolution Network (BP-SGCN) that learns pseudo-labels and informs to a trajectory predictor. For optimization, we propose a cascaded training scheme, in which we first learn the pseudo-labels in an unsupervised manner, and then perform end-to-end fine-tuning on the labels in the direction of increasing the trajectory prediction accuracy. Experiments show that our pseudo-labels effectively model different behavior clusters and improve trajectory prediction. Our proposed BP-SGCN outperforms existing methods using both pedestrian (ETH/UCY, pedestrian-only SDD) and heterogeneous agent datasets (SDD, Argoverse 1).

Figures (7)

• Figure 1: We propose the behavioral pseudo-labels learned from observed trajectories, effectively representing inter- and intra-type behavioral differences to improve pedestrian and heterogeneous trajectory prediction accuracy.
• Figure 2: Trajectory visualization on heterogeneous SSD dataset, where red, green and blue dots represent pedestrians, bikers and cars, respectively. (a) and (c) represent heterogeneous scenarios with all agent types, (b) and (d) represent the pedestrian-only scenarios commonly used by pedestrian trajectory predictions mangalam2020PECNetmangalam2021ynet by simply removing all non-pedestrian agents.
• Figure 3: The overview of BP-SGCN to learn the pseudo-labels for trajectory prediction, consisting of the deep unsupervised clustering module and the pseudo-label informed trajectory prediction module. We propose a cascaded optimization scheme to first learn pseudo-labels in an unsupervised manner, and then fine-tune them in an end-to-end manner with trajectory prediction supervision.
• Figure 4: The t-SNE visualization of pseudo-class clustering on SDD ($k$=6) during unsupervised deep clustering. (a) 0 epochs (initialized by k-means), (b) 200 epochs, (c) 800 epochs.
• Figure 5: Visualization of trajectory prediction on SDD of Semantic-STGCNN rainbow2021semanticStgcnn, Multiclass-SGCNruochen2022multiclassSGCN, and BP-SGCN (ours). Blue and red represent observed and ground-truth trajectories respectively, yellow represents the predicted trajectory and light-blue shade represents the predicted distribution.