Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TSC: Topology-Conditioned Stackelberg Coordination for Multi-Agent Reinforcement Learning in Interactive Driving

Xiaotong Zhang, Gang Xiong, Yuanjing Wang, Siyu Teng, Alois Knoll, Long Chen

TL;DR

Topology-conditioned Stackelberg Coordination (TSC), a learning framework for decentralized interactive driving under communication-free execution, extracts a time-varying directed priority graph from braid-inspired weaving relations between trajectories, thereby defining local leader-follower dependencies without constructing a global order of play.

Abstract

Safe and efficient autonomous driving in dense traffic is fundamentally a decentralized multi-agent coordination problem, where interactions at conflict points such as merging and weaving must be resolved reliably under partial observability. With only local and incomplete cues, interaction patterns can change rapidly, often causing unstable behaviors such as oscillatory yielding or unsafe commitments. Existing multi-agent reinforcement learning (MARL) approaches either adopt synchronous decision-making, which exacerbate non-stationarity, or depend on centralized sequencing mechanisms that scale poorly as traffic density increases. To address these limitations, we propose Topology-conditioned Stackelberg Coordination (TSC), a learning framework for decentralized interactive driving under communication-free execution, which extracts a time-varying directed priority graph from braid-inspired weaving relations between trajectories, thereby defining local leader-follower dependencies without constructing a global order of play. Conditioned on this graph, TSC endogenously factorizes dense interactions into graph-local Stackelberg subgames and, under centralized training and decentralized execution (CTDE), learns a sequential coordination policy that anticipates leaders via action prediction and trains followers through action-conditioned value learning to approximate local best responses, improving training stability and safety in dense traffic. Experiments across four dense traffic scenarios show that TSC achieves superior performance over representative MARL baselines across key metrics, most notably reducing collisions while maintaining competitive traffic efficiency and control smoothness.

TSC: Topology-Conditioned Stackelberg Coordination for Multi-Agent Reinforcement Learning in Interactive Driving

TL;DR

Topology-conditioned Stackelberg Coordination (TSC), a learning framework for decentralized interactive driving under communication-free execution, extracts a time-varying directed priority graph from braid-inspired weaving relations between trajectories, thereby defining local leader-follower dependencies without constructing a global order of play.

Abstract

Safe and efficient autonomous driving in dense traffic is fundamentally a decentralized multi-agent coordination problem, where interactions at conflict points such as merging and weaving must be resolved reliably under partial observability. With only local and incomplete cues, interaction patterns can change rapidly, often causing unstable behaviors such as oscillatory yielding or unsafe commitments. Existing multi-agent reinforcement learning (MARL) approaches either adopt synchronous decision-making, which exacerbate non-stationarity, or depend on centralized sequencing mechanisms that scale poorly as traffic density increases. To address these limitations, we propose Topology-conditioned Stackelberg Coordination (TSC), a learning framework for decentralized interactive driving under communication-free execution, which extracts a time-varying directed priority graph from braid-inspired weaving relations between trajectories, thereby defining local leader-follower dependencies without constructing a global order of play. Conditioned on this graph, TSC endogenously factorizes dense interactions into graph-local Stackelberg subgames and, under centralized training and decentralized execution (CTDE), learns a sequential coordination policy that anticipates leaders via action prediction and trains followers through action-conditioned value learning to approximate local best responses, improving training stability and safety in dense traffic. Experiments across four dense traffic scenarios show that TSC achieves superior performance over representative MARL baselines across key metrics, most notably reducing collisions while maintaining competitive traffic efficiency and control smoothness.
Paper Structure (24 sections, 3 theorems, 49 equations, 8 figures, 2 tables)

This paper contains 24 sections, 3 theorems, 49 equations, 8 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Task and Objective
  4. Topological Interaction Graph
  5. Method
  6. Topological Priority Formulation
  7. Agent-centric future-trajectory representation
  8. Weaving distance and soft pairwise topological priority
  9. Scalar priority field from pairwise probabilities
  10. Joint prediction and supervision of pairwise and scalar priorities
  11. Topology-conditioned Stackelberg Coordination Network (TSC-Net)
  12. Trajectory weaving for local priority graph inference
  13. Priority-Conditioned Local Decision Interface
  14. Stackelberg Coordination on the Topological Priority Graph
  15. Learning Objective and Training Procedure
  16. ...and 9 more sections

Key Result

Lemma 1

Assume the per-step reward is bounded in magnitude by $|r_i^t|\le R_{\max}$, so any discounted value function under these operators satisfies $\|V\|_\infty\le V_{\max}$ with $V_{\max}=R_{\max}/(1-\gamma)$. Suppose that for some $\varepsilon_T \ge 0$ we have for all bounded $V$ with $\|V\|_\infty\le V_{\max}$. Then

Figures (8)

  • Figure 1: Topology-conditioned Stackelberg coordination from trajectory weaving in multi-vehicle traffic.
  • Figure 2: The TSC-Net architecture. Ego and neighbor encoders construct an interaction representation $\mathbf{H}_i^t$, from which the TopoDecoder predicts pairwise priorities $\mathbf{\hat{p}}_{i\leftarrow j}^t$ and node scores $\mathbf{\hat{s}}_i^t$ to form a local priority graph. Guided by this graph, a TopK filter and TopoAttention aggregate the most influential neighbors into a compact decision state $u_i^t$, which the PolicyHead maps to the continuous action $a_i^t$. The same priority structure defines the local leader set $\mathcal{L}_i^t$. The PredictHead estimates leader actions $\{\hat{a}_{j\mid i}^t\}_{j\in\mathcal{L}_i^t}$, and the ValueHead conditions on these predictions to approximate the local Stackelberg value, yielding TD advantages $\hat{A}_i^t$ for actor optimization under CTDE.
  • Figure 3: Evaluation scenarios for dense multi-vehicle interaction.
  • Figure 4: Comparison of safety, efficiency, and smoothness across four scenarios.
  • Figure 5: Collision rate of different methods under varying numbers of vehicles at test time.
  • ...and 3 more figures

Theorems & Definitions (5)

  • Lemma 1: Bellman error bound
  • proof
  • Lemma 2: Performance difference lemma for the local Stackelberg MDP
  • proof
  • Corollary 1