Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Deep Reinforcement Learning for Energy Efficient Multi-Functional RIS-Assisted Low-Earth Orbit Networks

Li-Hsiang Shen, Jyun-Jhe Huang, Kai-Ten Feng, Lie-Liang Yang, Jen-Ming Wu

TL;DR

This work tackles enabling energy-efficient, globally covered LEO communications by deploying multi-functional RIS on satellites. It formulates a long-term EE optimization over MF-RIS configurations, EH ratios, and LEO beamforming, and resolves the non-convex, dynamic problem with a Federated Learning enhanced MADDPG (FEMAD) scheme. FEMAD combines per-agent MADDPG with FL-based parameter exchange to improve cooperation and convergence, achieving higher EE than centralized DRL or fully distributed methods. Results show MF-RIS energy harvesting and amplification are crucial for performance, and the approach scales effectively with network size while maintaining energy efficiency advantages over RIS-free or fixed-EH configurations.

Abstract

In this paper, a novel network architecture that deploys the multi-functional reconfigurable intelligent surface (MF-RIS) in low-Earth orbit (LEO) is proposed. Unlike traditional RIS with only signal reflection capability, the MF-RIS can reflect, refract, and amplify signals, as well as harvest energy from wireless signals. Given the high energy demands in shadow regions where solar energy is unavailable, MF-RIS is deployed in LEO to enhance signal coverage and improve energy efficiency (EE). To address this, we formulate a long-term EE optimization problem by determining the optimal parameters for MF-RIS configurations, including amplification and phase-shifts, energy harvesting ratios, and LEO transmit beamforming. To address the complex non-convex and non-linear problem, a federated learning enhanced multi-agent deep deterministic policy gradient (FEMAD) scheme is designed. Multi-agent DDPG of each agent can provide the optimal action policy from its interaction to environments, whereas federated learning enables the hidden information exchange among multi-agents. In numerical results, we can observe significant EE improvements compared to the other benchmarks, including centralized deep reinforcement learning as well as distributed multi-agent deep deterministic policy gradient (DDPG). Additionally, the proposed LEO-MF-RIS architecture has demonstrated its effectiveness, achieving the highest EE performance compared to the scenarios of fixed/no energy harvesting in MF-RIS, traditional reflection-only RIS, and deployment without RISs/MF-RISs.

Federated Deep Reinforcement Learning for Energy Efficient Multi-Functional RIS-Assisted Low-Earth Orbit Networks

TL;DR

This work tackles enabling energy-efficient, globally covered LEO communications by deploying multi-functional RIS on satellites. It formulates a long-term EE optimization over MF-RIS configurations, EH ratios, and LEO beamforming, and resolves the non-convex, dynamic problem with a Federated Learning enhanced MADDPG (FEMAD) scheme. FEMAD combines per-agent MADDPG with FL-based parameter exchange to improve cooperation and convergence, achieving higher EE than centralized DRL or fully distributed methods. Results show MF-RIS energy harvesting and amplification are crucial for performance, and the approach scales effectively with network size while maintaining energy efficiency advantages over RIS-free or fixed-EH configurations.

Abstract

In this paper, a novel network architecture that deploys the multi-functional reconfigurable intelligent surface (MF-RIS) in low-Earth orbit (LEO) is proposed. Unlike traditional RIS with only signal reflection capability, the MF-RIS can reflect, refract, and amplify signals, as well as harvest energy from wireless signals. Given the high energy demands in shadow regions where solar energy is unavailable, MF-RIS is deployed in LEO to enhance signal coverage and improve energy efficiency (EE). To address this, we formulate a long-term EE optimization problem by determining the optimal parameters for MF-RIS configurations, including amplification and phase-shifts, energy harvesting ratios, and LEO transmit beamforming. To address the complex non-convex and non-linear problem, a federated learning enhanced multi-agent deep deterministic policy gradient (FEMAD) scheme is designed. Multi-agent DDPG of each agent can provide the optimal action policy from its interaction to environments, whereas federated learning enables the hidden information exchange among multi-agents. In numerical results, we can observe significant EE improvements compared to the other benchmarks, including centralized deep reinforcement learning as well as distributed multi-agent deep deterministic policy gradient (DDPG). Additionally, the proposed LEO-MF-RIS architecture has demonstrated its effectiveness, achieving the highest EE performance compared to the scenarios of fixed/no energy harvesting in MF-RIS, traditional reflection-only RIS, and deployment without RISs/MF-RISs.
Paper Structure (10 sections, 24 equations, 6 figures, 1 table)

This paper contains 10 sections, 24 equations, 6 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. Channel Model
  4. Power Dissipation Model
  5. Problem Formulation
  6. Proposed Federated Learning Multi-Agent Deep Deterministic Policy Gradient (FEMAD) Scheme
  7. Multi-Agent DDPG Algorithm
  8. Enhancing MADDPG with Federated Learning
  9. Simulation result
  10. Conclusions

Figures (6)

  • Figure 1: The proposed architecture of LEO-MF-RIS. MF-RIS is equipped on LEO satellites, enabling hybrid signal and energy harvesting mode.
  • Figure 2: The proposed FEMAD architecture for LEO-MF-RIS networks.
  • Figure 3: Convergence of FEMAD and existing DRL methods.
  • Figure 4: EE with different MF-RIS configurations, i.e., no amplification, fixed/no energy harvesting, and no-RIS versus different numbers of LEOs.
  • Figure 5: EE versus different numbers of MF-RIS elements. A portion of elements are switched on, i.e., full-on, half-on, $30\%$ and $10\%$ of elements.
  • ...and 1 more figures