DyTopo: Dynamic Topology Routing for Multi-Agent Reasoning via Semantic Matching

TL;DR

We address fixed topology limitations in multi-agent reasoning with large language models by introducing DyTopo, which dynamically rewires a sparse directed graph $G^{(t)}$ at each round under a manager-specified goal. Agents emit a need descriptor $s^{(t)}_{q,i}$ and a key descriptor $s^{(t)}_{k,i}$; a semantic encoder computes $q_i^{(t)}$ and $k_j^{(t)}$, and edges are activated when $r_{i,j}^{(t)}> au_{edge}$ where $r_{i,j}^{(t)}=(\hat{q}_i^{(t)})^T\hat{k}_j^{(t)}$. A Manager meta-agent maintains global state $S_{global}^{(t)}$ and makes halting decisions based on a threshold $\\gamma_{success}$, enabling a bi-level feedback loop for round progression. Empirically, across code-generation and mathematical reasoning benchmarks and multiple backbones, DyTopo outperforms fixed or random topologies, while offering an interpretable view into how communication paths reconfigure throughout reasoning.

Abstract

Multi-agent systems built from prompted large language models can improve multi-round reasoning, yet most existing pipelines rely on fixed, trajectory-wide communication patterns that are poorly matched to the stage-dependent needs of iterative problem solving. We introduce DyTopo, a manager-guided multi-agent framework that reconstructs a sparse directed communication graph at each round. Conditioned on the manager's round goal, each agent outputs lightweight natural-language query (need) and \key (offer) descriptors; DyTopo embeds these descriptors and performs semantic matching, routing private messages only along the induced edges. Across code generation and mathematical reasoning benchmarks and four LLM backbones, DyTopo consistently outperforms over the strongest baseline (avg. +6.2). Beyond accuracy, DyTopo yields an interpretable coordination trace via the evolving graphs, enabling qualitative inspection of how communication pathways reconfigure across rounds.

Figures (4)

• Figure 1: Comparison of communication topologies. (A) Single-agent prompting. (B) Fixed-topology communication reused across rounds. (C) DyTopo dynamically rewires a directed agent graph each round based on the round goal and semantic relevance.
• Figure 2: DyTopo round-by-round routing via semantic matching. At each round $t$, each worker agent outputs a query and a key descriptor. A semantic matching module embeds these descriptors, computes pairwise similarity, and induces a directed graph $G^{(t)}$. Private messages produced at round $t$ are routed according to $G^{(t)}$ after a synchronization barrier and are appended to recipients' memories for round $t{+}1$. The Manager provides round goals and updates the next-round context, yielding a closed-loop adaptation across rounds.
• Figure 3: Performance change on communication rounds for HumanEval and Math-500. HumanEval achieves optimal performance (92.07%) at the 5th round, while Math-500 peaks (87.14%) at the 9th round, suggesting task-specific performances.
• Figure 4: DyTopo rewires communication over rounds. For one code-generation instance, we show three representative rounds (goal, induced directed graph $G^{(t)}$, and execution order). The topology transitions from broad, exploratory routing (Round 1) to verification-focused connections (Round 2), and finally to a sparse, dependency-minimal graph for producing the formatted final answer (Round 3).