Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multi-Agent Conditional Diffusion Model with Mean Field Communication as Wireless Resource Allocation Planner

Kechen Meng, Sinuo Zhang, Rongpeng Li, Xiangming Meng, Yansha Deng, Chan Wang, Ming Lei, Zhifeng Zhao

TL;DR

This work tackles scalable decentralized wireless resource allocation by combining model-based reinforcement learning with a diffusion-based world model for each agent. It introduces MA-CDMP, a conditional diffusion model planner that uses mean-field communication to approximate inter-agent interactions under the DTDE paradigm, aided by an inverse dynamics module for action generation. The authors prove theoretical bounds on distributional error introduced by MF-guided diffusion and demonstrate superior QoS and scalability over MARL baselines in high-fidelity wireless simulations. The results indicate that diffusion-based planning with MF coordination is a practical, robust approach for real-world wireless network optimization.

Abstract

In wireless communication systems, efficient and adaptive resource allocation plays a crucial role in enhancing overall Quality of Service (QoS). Compared to the conventional Model-Free Reinforcement Learning (MFRL) scheme, Model-Based RL (MBRL) first learns a generative world model for subsequent planning. The reuse of historical experience in MBRL promises more stable training behavior, yet its deployment in large-scale wireless networks remains challenging due to high-dimensional stochastic dynamics, strong inter-agent cooperation, and communication constraints. To overcome these challenges, we propose the Multi-Agent Conditional Diffusion Model Planner (MA-CDMP) for decentralized communication resource management. Built upon the Distributed Training with Decentralized Execution (DTDE) paradigm, MA-CDMP models each communication node as an autonomous agent and employs Diffusion Models (DMs) to capture and predict environment dynamics. Meanwhile, an inverse dynamics model guides action generation, thereby enhancing sample efficiency and policy scalability. Moreover, to approximate large-scale agent interactions, a Mean-Field (MF) mechanism is introduced as an assistance to the classifier in DMs. This design mitigates inter-agent non-stationarity and enhances cooperation with minimal communication overhead in distributed settings. We further theoretically establish an upper bound on the distributional approximation error introduced by the MF-based diffusion generation, guaranteeing convergence stability and reliable modeling of multi-agent stochastic dynamics. Extensive experiments demonstrate that MA-CDMP consistently outperforms existing MARL baselines in terms of average reward and QoS metrics, showcasing its scalability and practicality for real-world wireless network optimization.

Multi-Agent Conditional Diffusion Model with Mean Field Communication as Wireless Resource Allocation Planner

TL;DR

This work tackles scalable decentralized wireless resource allocation by combining model-based reinforcement learning with a diffusion-based world model for each agent. It introduces MA-CDMP, a conditional diffusion model planner that uses mean-field communication to approximate inter-agent interactions under the DTDE paradigm, aided by an inverse dynamics module for action generation. The authors prove theoretical bounds on distributional error introduced by MF-guided diffusion and demonstrate superior QoS and scalability over MARL baselines in high-fidelity wireless simulations. The results indicate that diffusion-based planning with MF coordination is a practical, robust approach for real-world wireless network optimization.

Abstract

In wireless communication systems, efficient and adaptive resource allocation plays a crucial role in enhancing overall Quality of Service (QoS). Compared to the conventional Model-Free Reinforcement Learning (MFRL) scheme, Model-Based RL (MBRL) first learns a generative world model for subsequent planning. The reuse of historical experience in MBRL promises more stable training behavior, yet its deployment in large-scale wireless networks remains challenging due to high-dimensional stochastic dynamics, strong inter-agent cooperation, and communication constraints. To overcome these challenges, we propose the Multi-Agent Conditional Diffusion Model Planner (MA-CDMP) for decentralized communication resource management. Built upon the Distributed Training with Decentralized Execution (DTDE) paradigm, MA-CDMP models each communication node as an autonomous agent and employs Diffusion Models (DMs) to capture and predict environment dynamics. Meanwhile, an inverse dynamics model guides action generation, thereby enhancing sample efficiency and policy scalability. Moreover, to approximate large-scale agent interactions, a Mean-Field (MF) mechanism is introduced as an assistance to the classifier in DMs. This design mitigates inter-agent non-stationarity and enhances cooperation with minimal communication overhead in distributed settings. We further theoretically establish an upper bound on the distributional approximation error introduced by the MF-based diffusion generation, guaranteeing convergence stability and reliable modeling of multi-agent stochastic dynamics. Extensive experiments demonstrate that MA-CDMP consistently outperforms existing MARL baselines in terms of average reward and QoS metrics, showcasing its scalability and practicality for real-world wireless network optimization.
Paper Structure (26 sections, 4 theorems, 47 equations, 8 figures, 3 tables, 2 algorithms)

This paper contains 26 sections, 4 theorems, 47 equations, 8 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Multi-Agent Reinforcement Learning for Resource Scheduling
  4. Diffusion-Based Resource Allocation in Communication Systems
  5. Diffusion Models for Multi-Agent Reinforcement Learning
  6. Theoretical Analysis of Distributional Error in Diffusion-Based Generation
  7. Preliminaries & System Model
  8. Preliminaries
  9. Multi-Agent Model-Based Reinforcement Learning
  10. Diffusion Models
  11. System Model
  12. Resource Allocation With MA-CDMP
  13. Overview of MA-CDMP
  14. Training of Core Components in MA-CDMP
  15. Implementation with MF-Enhanced Classifier Guidance
  16. ...and 11 more sections

Key Result

Lemma 1

Under Assumptions assump:smooth and assump:difference, for the MF communication mechanism, the drift term approximation error of the reverse-time SDE in Eq. eq:reverse model SDE admits an upper bound as

Figures (8)

  • Figure 1: Overview of the multi-node wireless topology with MF-TDMA-based MAC scheduling.
  • Figure 2: The illustration of the MA-CDMP algorithm for resource allocation in wireless communication networks.
  • Figure 3: Comparison of MA-CDMP with other methods in terms of average reward.
  • Figure 4: Comparison of MA-CDMP with MADiff in terms of average reward.
  • Figure 5: Comparison of MA-CDMP with MA-CDMP_w/o_MF under varying node number scenarios in terms of average reward.
  • ...and 3 more figures

Theorems & Definitions (7)

  • Lemma 1
  • proof
  • Theorem 1
  • Lemma 2
  • Lemma 3
  • proof
  • Remark