Reasoning Multi-Agent Behavioral Topology for Interactive Autonomous Driving

TL;DR

BeTop tackles uncertainty in multi-agent driving by introducing Behavioral Topology (BeTop), a braid-theory based topological supervision that captures consensual interactive patterns. BeTopNet jointly learns to predict BeTop topology and trajectories using a topology-guided transformer with imitative contingency planning to handle uncertainty. The key contributions are the BeTop formulation, the BeTopNet architecture with topology-guided local attention, and the imitative contingency learning objective, yielding state-of-the-art results on nuPlan and WOMD datasets. This work advances explainable, topologically-informed multi-agent decision making for autonomous driving and suggests directions for recursive, 3D, and end-to-end perception-enabled extensions.

Abstract

Autonomous driving system aims for safe and social-consistent driving through the behavioral integration among interactive agents. However, challenges remain due to multi-agent scene uncertainty and heterogeneous interaction. Current dense and sparse behavioral representations struggle with inefficiency and inconsistency in multi-agent modeling, leading to instability of collective behavioral patterns when integrating prediction and planning (IPP). To address this, we initiate a topological formation that serves as a compliant behavioral foreground to guide downstream trajectory generations. Specifically, we introduce Behavioral Topology (BeTop), a pivotal topological formulation that explicitly represents the consensual behavioral pattern among multi-agent future. BeTop is derived from braid theory to distill compliant interactive topology from multi-agent future trajectories. A synergistic learning framework (BeTopNet) supervised by BeTop facilitates the consistency of behavior prediction and planning within the predicted topology priors. Through imitative contingency learning, BeTop also effectively manages behavioral uncertainty for prediction and planning. Extensive verification on large-scale real-world datasets, including nuPlan and WOMD, demonstrates that BeTop achieves state-of-the-art performance in both prediction and planning tasks. Further validations on the proposed interactive scenario benchmark showcase planning compliance in interactive cases.

Figures (10)

• Figure 1: Multi-agent Behavioral Formulation. (a) A typical driving scenario in Arizona, US Ettinger_2021_ICCV; (b) Dense representation conducts scalable occupancy prediction jointly, but restrained reception leads to unbounded collisions with planning; (c) Sparse supervision derives multi-agent trajectories with multi-modalities, while it struggles with conflicts among integrated prediction and planning; (d) BeTop reasons future topological behaviors for all scene-agents through braids theory, funneling interactive eventual agents (in highlighted colors) and guiding compliant joint prediction and planning.
• Figure 2: BeTop formulation. Joint future trajectories are transformed to braid sets, and then form joint topology through intertwine indicators.
• Figure 3: The BeTopNet Architecture. BeTop establishes an integrated network for topological behavior reasoning, comprising three fundamentals. Scene encoder generates scene-aware attributes for agent $\mathbf{S}_A$ and map $\mathbf{S}_M$. Initialized by $\mathbf{S}_R$ and $\mathbf{Q}_A$, synergistic decoder reasons edge topology $\hat{e}^l_n$ and trajectories $\hat{\mathbf{Y}}^l_n$ iteratively from topology-guided local attention. Branched planning $\tau\in\hat{\mathbf{Y}}_1$ with predictions and topology are optimized jointly by imitative contingency learning.
• Figure 4: Results of different interactive agents number for local attention. We observe a convergence effect for the selection of $K$.
• Figure 5: Qualitative results of planning and prediction in nuPlan and WOMD. BeTopNet performs compliant reaction simulations in a) yielding for pedestrians; b) cruising in dense traffic. Interactive scenarios (c,d) further present the consistency of contingency learning. BeTopNet predicts both compliant marginal (e,f) and joint (g,h) multi-agent predictions under diverse scenarios. Future interactive behavior patterns can also be consistently reasoned (rendered in light red) with BeTop.

Theorems & Definitions (13)

• Theorem B.1: Geometric Invariant
• proof
• Remark 1
• Definition B.1: Topological Invariant
• proof
• Corollary B.1.1
• proof
• Theorem B.2
• proof
• Remark 2