Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synthesis of Communication Policies for Multi-Agent Systems Robust to Communication Restrictions

Saleh Soudijani, Rayna Dimitrova

TL;DR

This work addresses the challenge of coordinating multiple agents to maximize a joint reach-avoid probability when communication is restricted. It introduces a two-stage policy synthesis framework that first computes the optimal reach-avoid probability under full communication via an occupancy-measure linear program, and then minimizes an entropy-based information-sharing cost $\bar{D}_{(\pi_{comm},\pi_{act})}$ to produce a robust pair of policies that respect the communication limit $K$. The core novelty is the explicitly defined, information-theoretic cost $D_{(\pi_{comm},\pi_{act})}$ and its tractable proxy $\bar{D}$, which provide a formal bound on performance loss due to restricted communication and guide the synthesis toward zero-cost communication when feasible. The method is demonstrated on four three-agent navigation benchmarks, showing that zero-cost communication policies can achieve the unconstrained optimum in many cases and that dynamic communication policies adapt to public information. Overall, the approach offers a rigorous, scalable framework for robust coordination in MAS under bandwidth constraints with practical implications for decentralized control and robot swarms.

Abstract

We study stochastic multi-agent systems in which agents must cooperate to maximize the probability of achieving a common reach-avoid objective. In many applications, during the execution of the system, the communication between the agents can be constrained by restrictions on the bandwidth currently available for exchanging local-state information between the agents. In this paper, we propose a method for computing joint action and communication policies for the group of agents that aim to satisfy the communication restrictions as much as possible while achieving the optimal reach-avoid probability when communication is unconstrained. Our method synthesizes a pair of action and communication policies robust to restrictions on the number of agents allowed to communicate. To this end, we introduce a novel cost function that measures the amount of information exchanged beyond what the communication policy allows. We evaluate our approach experimentally on a range of benchmarks and demonstrate that it is capable of computing pairs of action and communication policies that satisfy the communication restrictions, if such exist.

Synthesis of Communication Policies for Multi-Agent Systems Robust to Communication Restrictions

TL;DR

This work addresses the challenge of coordinating multiple agents to maximize a joint reach-avoid probability when communication is restricted. It introduces a two-stage policy synthesis framework that first computes the optimal reach-avoid probability under full communication via an occupancy-measure linear program, and then minimizes an entropy-based information-sharing cost to produce a robust pair of policies that respect the communication limit . The core novelty is the explicitly defined, information-theoretic cost and its tractable proxy , which provide a formal bound on performance loss due to restricted communication and guide the synthesis toward zero-cost communication when feasible. The method is demonstrated on four three-agent navigation benchmarks, showing that zero-cost communication policies can achieve the unconstrained optimum in many cases and that dynamic communication policies adapt to public information. Overall, the approach offers a rigorous, scalable framework for robust coordination in MAS under bandwidth constraints with practical implications for decentralized control and robot swarms.

Abstract

We study stochastic multi-agent systems in which agents must cooperate to maximize the probability of achieving a common reach-avoid objective. In many applications, during the execution of the system, the communication between the agents can be constrained by restrictions on the bandwidth currently available for exchanging local-state information between the agents. In this paper, we propose a method for computing joint action and communication policies for the group of agents that aim to satisfy the communication restrictions as much as possible while achieving the optimal reach-avoid probability when communication is unconstrained. Our method synthesizes a pair of action and communication policies robust to restrictions on the number of agents allowed to communicate. To this end, we introduce a novel cost function that measures the amount of information exchanged beyond what the communication policy allows. We evaluate our approach experimentally on a range of benchmarks and demonstrate that it is capable of computing pairs of action and communication policies that satisfy the communication restrictions, if such exist.
Paper Structure (37 sections, 8 theorems, 31 equations, 11 figures, 5 tables)

This paper contains 37 sections, 8 theorems, 31 equations, 11 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work.
  3. Preliminaries
  4. Entropy of Stochastic Processes
  5. Multi-Agent Markov Decision Processes
  6. Problem Formulation
  7. Execution under Restricted Communication
  8. Policy Synthesis
  9. Cost Function for Information Sharing Relative to a Communication Policy
  10. Policy Synthesis
  11. Optimistic Optimal Value for Reach-Avoid Probability
  12. Minimizing the Cost of Communication
  13. Cost optimization problem
  14. Policies from an optimal solution
  15. Experimental Evaluation
  16. ...and 22 more sections

Key Result

Theorem 1

For any cooperative Markov game $\widehat{M}$ with MMDP $M$, reach-avoid objective $(\mathcal{S}_{\mathit{target}},\mathcal{S}_{\mathit{avoid}})$, and $\pi = (\pi_{comm},\pi_{act}) \in \Pi_{comm}^{pos}(\mathcal{O},K)\times\Pi^{pos}_{act}(M)$, it holds that

Figures (11)

  • Figure 1: Environment #1 for a robots navigation problem, with robots $R1$, $R2$, and $R3$ and their respective targets ($T1$, $T2$, $T3$).
  • Figure 2: Grid environments and regions labeled with public information for the scenarios in \ref{['sec:eval']}. Robots' initial positions are indicated by $R1$, $R2$, and $R3$, and their target positions by $T1$, $T2$, and $T3$.
  • Figure 3: Scenario #1. Heat maps of the occupancy measures for the policy computed without minimizing communication.
  • Figure 4: Scenario #1. Heat maps of the occupancy measures for the policy computed when minimizing communication.
  • Figure 5: Local states labels used in Scenario #2.
  • ...and 6 more figures

Theorems & Definitions (19)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Example 1
  • Definition 5
  • Theorem 1
  • Proposition 1
  • Proposition 2
  • Theorem 2
  • ...and 9 more