Stochastic Trajectory Prediction under Unstructured Constraints

TL;DR

This work tackles constrained trajectory prediction under unstructured constraints by introducing Controllable Trajectory Diffusion (CTD). CTD pairs a pre-trained scoring model, trained from pairwise constraint judgments via Bradley–Terry–Luce, with a conditional diffusion model that generates future trajectories conditioned on a constraint conformity score in $[0,1]$ and historical context. The approach avoids differentiable, hand-crafted constraint formulations and supports multiple, combinatorial constraints during inference, achieving competitive minADE and minFDE on ETH/UCY and SDD while aligning predictions with constraints such as speed and turning. By leveraging a constraint score as a conditioning signal, CTD enables controllable, semantically meaningful trajectory generation with potential for real-time planning in dynamic environments.

Abstract

Trajectory prediction facilitates effective planning and decision-making, while constrained trajectory prediction integrates regulation into prediction. Recent advances in constrained trajectory prediction focus on structured constraints by constructing optimization objectives. However, handling unstructured constraints is challenging due to the lack of differentiable formal definitions. To address this, we propose a novel method for constrained trajectory prediction using a conditional generative paradigm, named Controllable Trajectory Diffusion (CTD). The key idea is that any trajectory corresponds to a degree of conformity to a constraint. By quantifying this degree and treating it as a condition, a model can implicitly learn to predict trajectories under unstructured constraints. CTD employs a pre-trained scoring model to predict the degree of conformity (i.e., a score), and uses this score as a condition for a conditional diffusion model to generate trajectories. Experimental results demonstrate that CTD achieves high accuracy on the ETH/UCY and SDD benchmarks. Qualitative analysis confirms that CTD ensures adherence to unstructured constraints and can predict trajectories that satisfy combinatorial constraints.

Figures (4)

• Figure 1: The training pipeline of CTD. The encoder at the top left is responsible for encoding interaction and history information, and the module in the middle left receives $\mathbf{f}$ and ${\bm{y}}$ and gives the score $c$. The module at the bottom left is the parameterless encoder in diffusion. On the right is the diffusion model's decoder, which is responsible for recovering the original ${\bm{y}}$ from the noise.
• Figure 2: The inference phase of CTD. In the inference process, the predicted trajectory is obtained by iteratively denoising $T$ steps using the decoder $\epsilon_\theta$, conditioned on the encoding $\mathbf{f}$ and the assigned constraint $\tilde{c}$.
• Figure 3: Prediction results controlled by different constraints. At a T-intersection, visual elements demarcate distinct trajectory components: the historical trajectory in red, the true trajectory in blue, the model-sampled trajectory in green, and the distribution of predicted trajectories represented by the blue region. In subfigure (a), $\tilde{c}$ governs the directional bias of future trajectories, with increasing values of $\tilde{c}$ corresponding to a stronger constraint for rightward turns in the predicted trajectory. In subfigure (b), the influence of $\tilde{c}$ remains consistent, but larger values result in a deceleration of the predicted trajectory's speed.
• Figure 4: The combination of two constraints. The vertical axis represents different speed constraints, with larger controllable input and larger values resulting in slower trajectories. The horizontal axis represents different direction constraints, with larger controllable input and larger values resulting in trajectories that are more inclined towards turning left. During inference, two controllable inputs are concatenated as conditions for the diffusion model. The image shows that CTD can combine multiple constraints.