Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks

Xingran Chen, Navid NaderiAlizadeh, Alejandro Ribeiro, Shirin Saeedi Bidokhti

TL;DR

This work tackles real-time sampling and estimation in dynamic, multi-hop wireless networks by bridging graph neural networks with multi-agent reinforcement learning. The proposed graphical MARL framework combines a graphical actor-critic pair with a graph-based action distribution, enabling decentralized decisions that minimize the time-average estimation error. A key theoretical contribution is the transferability analysis, showing policies trained on structurally similar graphs generalize to larger networks via graphon-based arguments and WRNN-GRNN correspondence. Empirical results demonstrate superior performance over baselines, effective transfer to larger graphs, robustness to non-stationarity through recurrence and centralized training with decentralized execution, and notable gains in both synthetic and real network topologies. This work advances scalable, transferable decentralized learning for real-time estimation in dynamic wireless environments.

Abstract

We address real-time sampling and estimation of autoregressive Markovian sources in dynamic yet structurally similar multi-hop wireless networks. Each node caches samples from others and communicates over wireless collision channels, aiming to minimize time-average estimation error via decentralized policies. Due to the high dimensionality of action spaces and complexity of network topologies, deriving optimal policies analytically is intractable. To address this, we propose a graphical multi-agent reinforcement learning framework for policy optimization. Theoretically, we demonstrate that our proposed policies are transferable, allowing a policy trained on one graph to be effectively applied to structurally similar graphs. Numerical experiments demonstrate that (i) our proposed policy outperforms state-of-the-art baselines; (ii) the trained policies are transferable to larger networks, with performance gains increasing with the number of agents; (iii) the graphical training procedure withstands non-stationarity, even when using independent learning techniques; and (iv) recurrence is pivotal in both independent learning and centralized training and decentralized execution, and improves the resilience to non-stationarity.

Decentralized Learning Strategies for Estimation Error Minimization with Graph Neural Networks

TL;DR

This work tackles real-time sampling and estimation in dynamic, multi-hop wireless networks by bridging graph neural networks with multi-agent reinforcement learning. The proposed graphical MARL framework combines a graphical actor-critic pair with a graph-based action distribution, enabling decentralized decisions that minimize the time-average estimation error. A key theoretical contribution is the transferability analysis, showing policies trained on structurally similar graphs generalize to larger networks via graphon-based arguments and WRNN-GRNN correspondence. Empirical results demonstrate superior performance over baselines, effective transfer to larger graphs, robustness to non-stationarity through recurrence and centralized training with decentralized execution, and notable gains in both synthetic and real network topologies. This work advances scalable, transferable decentralized learning for real-time estimation in dynamic wireless environments.

Abstract

We address real-time sampling and estimation of autoregressive Markovian sources in dynamic yet structurally similar multi-hop wireless networks. Each node caches samples from others and communicates over wireless collision channels, aiming to minimize time-average estimation error via decentralized policies. Due to the high dimensionality of action spaces and complexity of network topologies, deriving optimal policies analytically is intractable. To address this, we propose a graphical multi-agent reinforcement learning framework for policy optimization. Theoretically, we demonstrate that our proposed policies are transferable, allowing a policy trained on one graph to be effectively applied to structurally similar graphs. Numerical experiments demonstrate that (i) our proposed policy outperforms state-of-the-art baselines; (ii) the trained policies are transferable to larger networks, with performance gains increasing with the number of agents; (iii) the graphical training procedure withstands non-stationarity, even when using independent learning techniques; and (iv) recurrence is pivotal in both independent learning and centralized training and decentralized execution, and improves the resilience to non-stationarity.
Paper Structure (10 sections, 2 theorems, 5 equations, 3 figures)

This paper contains 10 sections, 2 theorems, 5 equations, 3 figures.

Table of Contents

  1. Introduction
  2. System Model
  3. Proposed Graphical MARL Framework
  4. Graphical Actor and Critic
  5. Action Distribution
  6. Fundamental Analysis: Transferability
  7. Experimental Results
  8. Experimental Setup and Baselines
  9. Numerical Results
  10. Conclusion

Key Result

Theorem 1

Let $Y$ and $Y_m$ be defined in eq:PsiY and eq:PsiYm. Assume the input and output feature dimensions satisfy $F=G=1$, and define $\eta_1=\max_{1\leqslant t\leqslant T}\|X_t\|$ and $\eta_2=\max_{1\leqslant t\leqslant T}\|X_t-X_{t,m}\|$. Then, for any $0<\epsilon\leq 1$, it holds thatAs noted by SMRLG where $\Theta_1 = (\Omega+\frac{\pi\kappa^\epsilon_{W_{\Xi_m}}}{\delta^\epsilon_{WW_{\Xi_m}}})\|W -

Figures (3)

  • Figure 1: The proposed graphical MARL framework.
  • Figure 2: Performances of the proposed policies and baselines.
  • Figure 3: (a) Transferability in Watts–Strogatz networks: The policies are trained on $10$-node networks and tested on networks with $M\in[10, 50]$ nodes. (b) Performances of proposed policies in the real network under different $T$.

Theorems & Definitions (2)

  • Theorem 1: transferability in GRNNs
  • Theorem 2: Transferability in action distributions