Pedestrian Motion Prediction Using Transformer-based Behavior Clustering and Data-Driven Reachability Analysis

TL;DR

This work utilizes a transformer encoder coupled with hierarchical density-based clustering to automatically identify diverse behavior patterns, and shows that these behavior clusters can be used with data-driven reachability analysis, yielding an end-to-end data-driven approach to predicting the future motion of pedestrians.

Abstract

In this work, we present a transformer-based framework for predicting future pedestrian states based on clustered historical trajectory data. In previous studies, researchers propose enhancing pedestrian trajectory predictions by using manually crafted labels to categorize pedestrian behaviors and intentions. However, these approaches often only capture a limited range of pedestrian behaviors and introduce human bias into the predictions. To alleviate the dependency on manually crafted labels, we utilize a transformer encoder coupled with hierarchical density-based clustering to automatically identify diverse behavior patterns, and use these clusters in data-driven reachability analysis. By using a transformer-based approach, we seek to enhance the representation of pedestrian trajectories and uncover characteristics or features that are subsequently used to group trajectories into different "behavior" clusters. We show that these behavior clusters can be used with data-driven reachability analysis, yielding an end-to-end data-driven approach to predicting the future motion of pedestrians. We train and evaluate our approach on a real pedestrian dataset, showcasing its effectiveness in forecasting pedestrian movements.

Figures (7)

• Figure 1: Overview of the proposed framework
• Figure 2: Framework Architecture The figure displays the three main components of the proposed framework: (a) Trajectory Encoding Training Process, which involves the preparation and encoding of the input trajectories to extract informative features; (b) the Pedestrian Behavior Clustering for grouping the encoded embeddings based on their similarity; (c) Data-Driven Reachability Analysis of pedestrians using data of the same cluster
• Figure 3: Trajectory Encoding Model: The figure illustrates the encoding process of three pedestrian trajectory instances segmented to 50 time points, with six features included at each time point: (a) Transformer Encoder of trajectory data; (b) the Autoregression Training Task for unsupervised training of the transformer encoder.
• Figure 4: Transformer encoder training and validation loss.
• Figure 5: Reachable sets for three different scenarios using historical trajectories based on: the baseline and labeling oracle defined by Söderlund2023, a non-encoded trajectory clustering, and a transformer-encoded trajectory clustering approach.