Kinematics-aware Trajectory Generation and Prediction with Latent Stochastic Differential Modeling

TL;DR

This work addresses the challenge of producing physically realistic and controllable vehicle trajectories for autonomous driving by integrating kinematic physics into a learning framework. It introduces a latent kinematics-aware SDE (LK-SDE) within a variational autoencoder, where a graph-based context representation informs a latent SDE guided by a learnable bicycle-model drift; a decoder then produces trajectory outputs. Key contributions include the dual SDE design (a learnable bicycle-model SDE and a neural LK-SDE) with a kin-loss that aligns latent dynamics to physics, KL regularization of the latent initial state, and a robust optimization pipeline that yields smoother, more realistic trajectories and accurate latent-variable predictions. Empirical results on Argoverse show reduced jerk and improved alignment with real acceleration distributions, while maintaining competitive trajectory prediction accuracy and enabling estimation of unobservable kinematic states, enabling safer scenario augmentation and planning.

Abstract

Trajectory generation and trajectory prediction are two critical tasks in autonomous driving, which generate various trajectories for testing during development and predict the trajectories of surrounding vehicles during operation, respectively. In recent years, emerging data-driven deep learning-based methods have shown great promise for these two tasks in learning various traffic scenarios and improving average performance without assuming physical models. However, it remains a challenging problem for these methods to ensure that the generated/predicted trajectories are physically realistic. This challenge arises because learning-based approaches often function as opaque black boxes and do not adhere to physical laws. Conversely, existing model-based methods provide physically feasible results but are constrained by predefined model structures, limiting their capabilities to address complex scenarios. To address the limitations of these two types of approaches, we propose a new method that integrates kinematic knowledge into neural stochastic differential equations (SDE) and designs a variational autoencoder based on this latent kinematics-aware SDE (LK-SDE) to generate vehicle motions. Experimental results demonstrate that our method significantly outperforms both model-based and learning-based baselines in producing physically realistic and precisely controllable vehicle trajectories. Additionally, it performs well in predicting unobservable physical variables in the latent space.

Figures (7)

• Figure 1: Generating diverse, physically realistic, and controllable trajectories is important for critical scenario augmentation and vehicle motion prediction.
• Figure 2: The overall design of our kinematics-aware trajectory generator. The GCN and two individual encoders consume the driving historical trajectories $o_{i-k:i}$ and map information $\mathcal{M}$. They extract and generate the latent initial state $z_0$, global semantic feature $\vb{sem}$, and context feature per step $\vb{ctx}_t$ for the latent space. Within the latent space, we learn a kinematics-aware neural SDE guided by a physical bicycle model and then decode the latent vectors $\vb{z}_{i} (i=0, \cdots, T)$ to the output vehicle motion trajectory $\hat{o}_{i: i + T}$. Our neural LK-SDE is guided by the kinematic bicycle model during training to learn physical knowledge for physically feasible and controllable trajectory generation and prediction.
• Figure 3: During the training, in the latent space, the bicycle model SDE guides our neural LK-SDE to follow the kinematics by minimizing the KL divergence between the solutions of two SDEs. The kinematic loss function $L_{kin}$ is explained in Eq. \ref{['eq:kinematic_loss']}. $f_{\theta_0}$, $g_{\theta_1}$ are the neural networks for LK-SDE that are optimized from the bicycle model in Eq. \ref{['eq:kinematic_loss']} and output loss function in Eq. \ref{['eq:l1-smooth']}, while $\pi$ in the bicycle model is optimized by the output loss function in Eq. \ref{['eq:l1-smooth']}.
• Figure 4: Illustration of the bicycle modelpolack2017kinematic.
• Figure 5: The distribution of the jerk magnitude of different generative methods. The red dashed line represents the discomfort threshold for jerk value. We notice that our proposed LK-SDE can augment the smoothest trajectories.

Theorems & Definitions (2)

• Definition 1
• Definition 2