Towards Optimal Performance and Action Consistency Guarantees in Dec-POMDPs with Inconsistent Beliefs and Limited Communication

TL;DR

The paper tackles decentralized decision-making in Dec-POMDPs where agents hold inconsistent beliefs due to limited communication. It introduces Dec-OAC-POMDP-OL, a decentralized open-loop planner that provides probabilistic guarantees for Multi-Robot Optimal Action Consistency (MROAC) and can selectively trigger communication to improve inference performance, while also determining whether data sharing is beneficial after joint action selection. It formalizes how agents reason about unshared information, derives probabilistic guarantees for coordination, and proposes strategies (e.g., delta-NEPG) to trigger communication based on the expected impact on performance. Simulation in a collaborative fire-detection domain demonstrates that the approach can outperform state-of-the-art open-loop Dec-POMDP planners and closely match MPOMDP-OL performance with controlled communication, offering practical gains for safe, scalable multi-agent operation.

Abstract

Multi-agent decision-making under uncertainty is fundamental for effective and safe autonomous operation. In many real-world scenarios, each agent maintains its own belief over the environment and must plan actions accordingly. However, most existing approaches assume that all agents have identical beliefs at planning time, implying these beliefs are conditioned on the same data. Such an assumption is often impractical due to limited communication. In reality, agents frequently operate with inconsistent beliefs, which can lead to poor coordination and suboptimal, potentially unsafe, performance. In this paper, we address this critical challenge by introducing a novel decentralized framework for optimal joint action selection that explicitly accounts for belief inconsistencies. Our approach provides probabilistic guarantees for both action consistency and performance with respect to open-loop multi-agent POMDP (which assumes all data is always communicated), and selectively triggers communication only when needed. Furthermore, we address another key aspect of whether, given a chosen joint action, the agents should share data to improve expected performance in inference. Simulation results show our approach outperforms state-of-the-art algorithms.

Figures (3)

• Figure 1: Example of two agents in a collaborative fire detection task. Figure \ref{['fig:motivation-example-layout']} shows the initial layout and the possible movements of the agents. Each agent holds an observation of her current cell (in orange), which indicates with high certainty the cell is Empty. The agents did not share these observations, which makes the agents' beliefs inconsistent. Figures \ref{['fig:motivation-example-r-selection']} and \ref{["fig:motivation-example-r'-selection"]} show the agent's inconsistent beliefs (blue circles represent the amount of uncertainty of a cell). Each agent, based on the information available to her, is highly certain about her cell and highly uncertain about the other agent's cell. So, both agents consistently select each agent moves to the other agent's cell, thus satisfying MRAC. On the other hand, Figure \ref{['fig:motivation-example-full-history-selection']} shows that when considering all the data in the system, i.e. in an MPOMDP setting, the uncertainty of cell C is higher than the uncertainties of cells A and B, thus both agents consistently select both of them to move down and observe cell C.
• Figure 2: Figure \ref{['fig:local-history-r']} shows the available history of agent $r$ in bold (${\prescript{c}{}{\mathrm{h}}_{k}^{r,r'}}$, ${\Delta \mathrm{h}_{k}^{r,r'}}$), and the available history of agent $r'$ in dashed (${\prescript{c}{}{\mathrm{h}}_{k}^{r,r'}}$, ${\Delta \mathrm{h}_{k}^{r',r}}$), the full joint history ${\mathrm{h}_{k}^{\mathbb{D}}}$ is the union of them. Figure \ref{['fig:possible-full-history-r']} shows possibilities of full joint histories ${\tilde{h}_{k}^{\mathbb{D}}}$ from the perspective of agent $r$, by reasoning over the unshared data of agent $r'$, ${\Delta \tilde{h}_{k}^{r'}}$. Figure \ref{['fig:possible-full-history-c']} shows possibilities of full joint histories ${\tilde{h}_{k}^{\mathbb{D}}}$ by reasoning over the unshared data of agents $r$ and $r'$, $({\Delta \tilde{h}_{k}^{r'}}, {\Delta \tilde{h}_{k}^{r}})$.
• Figure 3: Illustration of Dec-OAC-POMDP-OL from a specific run in a 2x2 grid scenario. Figure \ref{['plot:mloas-opt-action-dist-reapet1-1step-00observations']} shows the $\epsilon$-MLOAS strategy with $\epsilon=0.3$, where action (D+D) is the selected optimal joint action. Figure \ref{['plot:mloas-mrac-dist-reapet1-1step-00observations']} shows the distribution of action selection by $r'$ in the calculations of MRAC guarantee $\epsilon=0.3$. Figure \ref{['plot:mloas-performance-gap-reapet1-1step-00observations']} shows the performance gap distribution of joint action (D+D) in the $\delta$-NEPG strategy, where the red line is the normalized expected performance gap. For $\delta=0.15$ the gap is withing the threshold, and for $\delta=0.05$ the gap is beyond the threshold, which triggers communication.

Theorems & Definitions (10)

• Definition III.1
• Definition III.2
• Definition IV.1
• Proposition IV.2
• proof
• Definition IV.3
• Definition IV.4
• Lemma IV.5
• Definition IV.6
• Lemma IV.7