Learning a Pedestrian Social Behavior Dictionary

TL;DR

While many trajectory analysis methods rely on RNNs or transformers, this work develops a lightweight, low-parameter approach and shows results comparable to SOTA on the ETH and UCY datasets.

Abstract

Understanding pedestrian behavior patterns is a key component to building autonomous agents that can navigate among humans. We seek a learned dictionary of pedestrian behavior to obtain a semantic description of pedestrian trajectories. Supervised methods for dictionary learning are impractical since pedestrian behaviors may be unknown a priori and the process of manually generating behavior labels is prohibitively time consuming. We instead utilize a novel, unsupervised framework to create a taxonomy of pedestrian behavior observed in a specific space. First, we learn a trajectory latent space that enables unsupervised clustering to create an interpretable pedestrian behavior dictionary. We show the utility of this dictionary for building pedestrian behavior maps to visualize space usage patterns and for computing the distributions of behaviors. We demonstrate a simple but effective trajectory prediction by conditioning on these behavior labels. While many trajectory analysis methods rely on RNNs or transformers, we develop a lightweight, low-parameter approach and show results comparable to SOTA on the ETH and UCY datasets.

Figures (7)

• Figure 1: Predicting pedestrian trajectories becomes easier when there is an understanding of the underlying social behaviors taking place. We create PT-Net to predict these social behaviors given a set of historical pedestrian locations. Each behavior maps to a different location in an embedding space, which allows for the characterization of a spectrum of pedestrian behavior.
• Figure 2: To make the pedestrian behavior dictionary, trajectories for small groups of nearby pedestrians are extracted from the datasets in overlapping windows of a fixed length. Velocity-based and proximity-based features are computed from these trajectories. During training, these processed trajectories are input to the t-SNE algorithm to create a 2D trajectory latent space embedding. PT-net takes in the processed trajectories and directly predicts the corresponding latent space coordinates from the ground truth t-SNE embedding. This learned coordinate embedding separates distinct pedestrian behavior into clusters in the space, each of which comprises a social behavior to form a pedestrian behavior dictionary. During inference, the processed trajectories are directly input to PT-net to get the embedding coordinates which are matched to the closest social behavior cluster.
• Figure 3: For pedestrian trajectory prediction, PT-net predicts the social behavior cluster assignment corresponding to the behavior of the pedestrians in the scene. This assignment dictates which of the MLPs will be used to predict the future trajectories of the pedestrians. We train one MLP per cluster in the pedestrian behavior dictionary and deterministically condition the prediction upon the social behaviors of the pedestrians in the scene.
• Figure 4: Each dot represents the t-SNE embedding of a 3.2 second trajectory. The colored clusters denote distinct social behaviors for N=1,2,3 people (corresponding colors across the three graphs do not denote related behaviors) along with trajectory diagrams, showing simplified sketches of the behaviors from Table \ref{['tab:behaviorTaxonomy']}.
• Figure 5: Pedestrian Behavior maps. Colored boxes indicate select pedestrian behaviors occurring over the entirety of the data collection period for ETH. We can infer a rich story about environment usage for varying numbers of pedestrians (N) and social behavior clusters (C). (\ref{['fig:N1C1/9Heatmap']}) (Green) Person entering the building; (Orange) Person leaving the building. Notice that people leave the building in a more narrowed and constrained path, indicating they are giving right-of-way to those entering. (\ref{['fig:N2C6/22Heatmap']}) (Yellow) Two people standing still together; (Red) Two people leaving the building to the left in a leader-follower formation. Notice that the people standing still tend to congregate off to the sides, or at an island in the middle of the walkway, and the people exiting fit into the gaps left behind. (\ref{['fig:N3C5/8Heatmap']}) (Purple) Two people walking side by side to exit the building passing one person entering the building; (Blue) Three people standing still. Notice there is a bottleneck around the door that prevents pedestrians from moving, but it becomes easier to move freely further from the door.