Heterogeneous Interaction Modeling With Reduced Accumulated Error for Multi-Agent Trajectory Prediction

TL;DR

This article proposes heterogeneous interaction modeling with reduced accumulated error (HIMRAE) for multiagent trajectory prediction, and proposes to introduce the graph entropy and the mixup training strategy for reducing the two types of errors.

Abstract

Dynamical complex systems composed of interactive heterogeneous agents are prevalent in the world, including urban traffic systems and social networks. Modeling the interactions among agents is the key to understanding and predicting the dynamics of the complex system, e.g., predicting the trajectories of traffic participants in the city. Compared with interaction modeling in homogeneous systems such as pedestrians in a crowded scene, heterogeneous interaction modeling is less explored. Worse still, the error accumulation problem becomes more severe since the interactions are more complex. To tackle the two problems, this paper proposes heterogeneous interaction modeling with reduced accumulated error for multi-agent trajectory prediction. Based on the historical trajectories, our method infers the dynamic interaction graphs among agents, featured by directed interacting relations and interacting effects. A heterogeneous attention mechanism is defined on the interaction graphs for aggregating the influence from heterogeneous neighbors to the target agent. To alleviate the error accumulation problem, this paper analyzes the error sources from the spatial and temporal perspectives, and proposes to introduce the graph entropy and the mixup training strategy for reducing the two types of errors respectively. Our method is examined on three real-world datasets containing heterogeneous agents, and the experimental results validate the superiority of our method.

Figures (10)

• Figure 1: A heterogeneous dynamical system involving three pedestrians (A, B and C) and one cyclist (D). Two snapshots of the trajectories are taken at time $t=3$ (a) and $t=6$ (b). A possible directed interaction graph among all traffic participants is provided (c).
• Figure 2: Overview of HIMRAE. HIMRAE takes the form of an auto-encoder. The encoder applies a GNN with a GRU over the historical trajectories to infer dynamic interaction graphs. The interaction graph is an edge-featured graph, where an edge represents a directed interacting relation and an edge feature represents the interacting effect. The graph entropy is used to control the structural complexity of the interaction graph. The decoder defines a heterogeneous attention mechanism (HAM) to model the interactions among heterogeneous agents, and predicts future trajectories recursively. The predicted trajectories and the ground truth trajectories are mixed with a coefficient $\lambda$ sampled from a beta distribution $\text{Beta}(\alpha,\alpha)$ to reduce the temporally accumulated error.
• Figure 3: (a) Visualization of the error sources from both the spatial and temporal perspectives. (b) A lower graph entropy favors a simplified interaction graph. (c) The predicted position and the ground truth position at time $t$ are mixed up to correct the multi-step prediction.
• Figure 4: For matrices in Subfig. (a)-(c), lighter indicates stronger connections. In Subfig. (d), historical trajectories are in dots, ground truth trajectories to be predicted are in solid lines, while the predicted trajectories are visualized using the kernel density estimation.
• Figure 5: Reduction of graph entropy at different graph density. Graphs inferred by HIMRAE and HIMRAE w/ GE are in blue and green, respectively.

Theorems & Definitions (3)

• Theorem 1: Informal
• Corollary 1
• Theorem 2