Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Multi-Agent DRL for Radio Resource Management in Industrial 6G in-X subnetworks

Bjarke Madsen, Ramoni Adeogun

TL;DR

The paper tackles interference management in dense industrial 6G in-X subnetworks by proposing privacy-preserving federated MARL solutions. It formulates the problem as a DEC-POMDP and develops two algorithms, F-MADDQN and F-MAPPO, that couple multi-agent learning with horizontal federated averaging to exchange only local model weights. Detailed designs for action/observation spaces and reward shaping are provided, along with training via FedAvg and an aggregation interval to balance convergence and signaling. Evaluation in a realistic indoor factory setting demonstrates that the federated approaches achieve competitive performance with reduced signaling overhead and strong robustness to deployment density and environment changes. The work offers practical, privacy-aware strategies for scalable RRM in industrial 6G scenarios, with implications for real-world deployments.

Abstract

Recently, 6G in-X subnetworks have been proposed as low-power short-range radio cells to support localized extreme wireless connectivity inside entities such as industrial robots, vehicles, and the human body. Deployment of in-X subnetworks within these entities may result in rapid changes in interference levels and thus, varying link quality. This paper investigates distributed dynamic channel allocation to mitigate inter-subnetwork interference in dense in-factory deployments of 6G in-X subnetworks. This paper introduces two new techniques, Federated Multi-Agent Double Deep Q-Network (F-MADDQN) and Federated Multi-Agent Deep Proximal Policy Optimization (F-MADPPO), for channel allocation in 6G in-X subnetworks. These techniques are based on a client-to-server horizontal federated reinforcement learning framework. The methods require sharing only local model weights with a centralized gNB for federated aggregation thereby preserving local data privacy and security. Simulations were conducted using a practical indoor factory environment proposed by 5G-ACIA and 3GPP models for in-factory environments. The results showed that the proposed methods achieved slightly better performance than baseline schemes with significantly reduced signaling overhead compared to the baseline solutions. The schemes also showed better robustness and generalization ability to changes in deployment densities and propagation parameters.

Federated Multi-Agent DRL for Radio Resource Management in Industrial 6G in-X subnetworks

TL;DR

The paper tackles interference management in dense industrial 6G in-X subnetworks by proposing privacy-preserving federated MARL solutions. It formulates the problem as a DEC-POMDP and develops two algorithms, F-MADDQN and F-MAPPO, that couple multi-agent learning with horizontal federated averaging to exchange only local model weights. Detailed designs for action/observation spaces and reward shaping are provided, along with training via FedAvg and an aggregation interval to balance convergence and signaling. Evaluation in a realistic indoor factory setting demonstrates that the federated approaches achieve competitive performance with reduced signaling overhead and strong robustness to deployment density and environment changes. The work offers practical, privacy-aware strategies for scalable RRM in industrial 6G scenarios, with implications for real-world deployments.

Abstract

Recently, 6G in-X subnetworks have been proposed as low-power short-range radio cells to support localized extreme wireless connectivity inside entities such as industrial robots, vehicles, and the human body. Deployment of in-X subnetworks within these entities may result in rapid changes in interference levels and thus, varying link quality. This paper investigates distributed dynamic channel allocation to mitigate inter-subnetwork interference in dense in-factory deployments of 6G in-X subnetworks. This paper introduces two new techniques, Federated Multi-Agent Double Deep Q-Network (F-MADDQN) and Federated Multi-Agent Deep Proximal Policy Optimization (F-MADPPO), for channel allocation in 6G in-X subnetworks. These techniques are based on a client-to-server horizontal federated reinforcement learning framework. The methods require sharing only local model weights with a centralized gNB for federated aggregation thereby preserving local data privacy and security. Simulations were conducted using a practical indoor factory environment proposed by 5G-ACIA and 3GPP models for in-factory environments. The results showed that the proposed methods achieved slightly better performance than baseline schemes with significantly reduced signaling overhead compared to the baseline solutions. The schemes also showed better robustness and generalization ability to changes in deployment densities and propagation parameters.
Paper Structure (21 sections, 17 equations, 6 figures, 1 table)

This paper contains 21 sections, 17 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. MARL FOR RRM IN SUBNETWORKS
  4. MARL Algorithms
  5. MADDQN
  6. MAPPO
  7. Proposed Methods
  8. Action Space
  9. Observation Space
  10. Reward Signal
  11. Policy
  12. Training
  13. PERFORMANCE EVALUATION
  14. Simulation Settings
  15. Benchmarks
  16. ...and 6 more sections

Figures (6)

  • Figure 1: Illustration of the training procedures for (a) multi-agent DDQN and (b) multi-agent PPO and the federated learning concept (c).
  • Figure 2: Illustration of the (a) Deep Q-Network (DQN) architecture for MADDQN and (b) Deep PPO architecture for MAPPO which are used for the simulations.
  • Figure 3: Averaged reward versus number of episodes.
  • Figure 4: CDF of achieved rate.
  • Figure 5: Sensitivity evaluation.
  • ...and 1 more figures