MATRIX: Multi-Agent Trajectory Generation with Diverse Contexts

TL;DR

This work proposes a learning-based automatic trajectory generation model, which it calls Multi-Agent TRajectory generation with dIverse conteXts (MATRIX), and achieves the goal of modeling the explicit and interpretable objectives so that MATRIX can generate human motions based on diverse destinations and heterogeneous behaviors.

Abstract

Data-driven methods have great advantages in modeling complicated human behavioral dynamics and dealing with many human-robot interaction applications. However, collecting massive and annotated real-world human datasets has been a laborious task, especially for highly interactive scenarios. On the other hand, algorithmic data generation methods are usually limited by their model capacities, making them unable to offer realistic and diverse data needed by various application users. In this work, we study trajectory-level data generation for multi-human or human-robot interaction scenarios and propose a learning-based automatic trajectory generation model, which we call Multi-Agent TRajectory generation with dIverse conteXts (MATRIX). MATRIX is capable of generating interactive human behaviors in realistic diverse contexts. We achieve this goal by modeling the explicit and interpretable objectives so that MATRIX can generate human motions based on diverse destinations and heterogeneous behaviors. We carried out extensive comparison and ablation studies to illustrate the effectiveness of our approach across various metrics. We also presented experiments that demonstrate the capability of MATRIX to serve as data augmentation for imitation-based motion planning.

Figures (5)

• Figure 1: Illustration of the MATRIX generation process. Based on the observed trajectory and sampled destinations, MATRIX can generate various heterogeneous trajectories through residual connection.
• Figure 2: The architecture of MATRIX. MATRIX consists of a Spatial-Temporal Encoder, a Gaussian Mixture Model (GMM), and a Gated Recurrent Unit (GRU) Decoder with a residual layer.
• Figure 3: Visualization of generated trajectories. Provided with the past trajectory (red), MATRIX (cyan) and Trajectron++ (green) can generate 20 possible future trajectories for five different scenes. We see that our generated trajectories are much more diverse than Trajectron++. Zoom in for better visualization.
• Figure 4: Visualization of GMM. We use MATRIX to generate five future trajectories (orange, cyan, violet, dark red, and royal blue) based on the past trajectory (red) and five stochastic destinations, represented as triangles, sampled from GMM. The center of each ellipse (green, magenta, yellow, and dark blue) is the mean of each Gaussian, and the radius is its one standard deviation. The weight of each Gaussian can be found in the legend. Zoom in for better visualization.
• Figure 5: Physics primitives of the generated data. We plot the histogram of physics primitives of four generated datasets -- MATRIX data (red), Agentformer Data (pink), Trajectron++ Data (blue), and Imitation Learning Data (yellow) -- against the raw one (green). The orange line is the best-fit curve. Note that we use exponential distribution for velocity and acceleration and Gaussian distribution for angular velocity and angular acceleration.