Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhanced Evolutionary Multi-Objective Deep Reinforcement Learning for Reliable and Efficient Wireless Rechargeable Sensor Networks

Bowei Tong, Hui Kang, Jiahui Li, Geng Sun, Jiacheng Wang, Yaoqi Yang, Bo Xu, Dusit Niyato

TL;DR

This paper tackles the NP-hard problem of jointly optimizing node survival and mobile charger energy efficiency in wireless rechargeable sensor networks over long horizons. It introduces an enhanced evolutionary multi-objective DRL framework (EMODRL) with an LSTM-based PPO policy, a MLP-based prospective increment model, and a time-varying Pareto policy evaluation mechanism (EMOPPO-TML) to learn diverse Pareto-optimal strategies. The approach reformulates the problem as a MOMDP and demonstrates superior convergence, robustness to energy-rate variations, and stronger policy diversity compared to baselines. The results indicate practical gains in reliability and efficiency for WRSNs under realistic mobility and energy constraints, enabling adaptive, autonomous charger management in remote environments.

Abstract

Despite rapid advancements in sensor networks, conventional battery-powered sensor networks suffer from limited operational lifespans and frequent maintenance requirements that severely constrain their deployment in remote and inaccessible environments. As such, wireless rechargeable sensor networks (WRSNs) with mobile charging capabilities offer a promising solution to extend network lifetime. However, WRSNs face critical challenges from the inherent trade-off between maximizing the node survival rates and maximizing charging energy efficiency under dynamic operational conditions. In this paper, we investigate a typical scenario where mobile chargers move and charge the sensor, thereby maintaining the network connectivity while minimizing the energy waste. Specifically, we formulate a multi-objective optimization problem that simultaneously maximizes the network node survival rate and mobile charger energy usage efficiency across multiple time slots, which presents NP-hard computational complexity with long-term temporal dependencies that make traditional optimization approaches ineffective. To address these challenges, we propose an enhanced evolutionary multi-objective deep reinforcement learning algorithm, which integrates a long short-term memory (LSTM)-based policy network for temporal pattern recognition, a multilayer perceptron-based prospective increment model for future state prediction, and a time-varying Pareto policy evaluation method for dynamic preference adaptation. Extensive simulation results demonstrate that the proposed algorithm significantly outperforms existing approaches in balancing node survival rate and energy efficiency while generating diverse Pareto-optimal solutions. Moreover, the LSTM-enhanced policy network converges 25% faster than conventional networks, with the time-varying evaluation method effectively adapting to dynamic conditions.

Enhanced Evolutionary Multi-Objective Deep Reinforcement Learning for Reliable and Efficient Wireless Rechargeable Sensor Networks

TL;DR

This paper tackles the NP-hard problem of jointly optimizing node survival and mobile charger energy efficiency in wireless rechargeable sensor networks over long horizons. It introduces an enhanced evolutionary multi-objective DRL framework (EMODRL) with an LSTM-based PPO policy, a MLP-based prospective increment model, and a time-varying Pareto policy evaluation mechanism (EMOPPO-TML) to learn diverse Pareto-optimal strategies. The approach reformulates the problem as a MOMDP and demonstrates superior convergence, robustness to energy-rate variations, and stronger policy diversity compared to baselines. The results indicate practical gains in reliability and efficiency for WRSNs under realistic mobility and energy constraints, enabling adaptive, autonomous charger management in remote environments.

Abstract

Despite rapid advancements in sensor networks, conventional battery-powered sensor networks suffer from limited operational lifespans and frequent maintenance requirements that severely constrain their deployment in remote and inaccessible environments. As such, wireless rechargeable sensor networks (WRSNs) with mobile charging capabilities offer a promising solution to extend network lifetime. However, WRSNs face critical challenges from the inherent trade-off between maximizing the node survival rates and maximizing charging energy efficiency under dynamic operational conditions. In this paper, we investigate a typical scenario where mobile chargers move and charge the sensor, thereby maintaining the network connectivity while minimizing the energy waste. Specifically, we formulate a multi-objective optimization problem that simultaneously maximizes the network node survival rate and mobile charger energy usage efficiency across multiple time slots, which presents NP-hard computational complexity with long-term temporal dependencies that make traditional optimization approaches ineffective. To address these challenges, we propose an enhanced evolutionary multi-objective deep reinforcement learning algorithm, which integrates a long short-term memory (LSTM)-based policy network for temporal pattern recognition, a multilayer perceptron-based prospective increment model for future state prediction, and a time-varying Pareto policy evaluation method for dynamic preference adaptation. Extensive simulation results demonstrate that the proposed algorithm significantly outperforms existing approaches in balancing node survival rate and energy efficiency while generating diverse Pareto-optimal solutions. Moreover, the LSTM-enhanced policy network converges 25% faster than conventional networks, with the time-varying evaluation method effectively adapting to dynamic conditions.
Paper Structure (35 sections, 1 theorem, 21 equations, 9 figures, 1 table, 5 algorithms)

This paper contains 35 sections, 1 theorem, 21 equations, 9 figures, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Charging Mechanism in WRSNs
  4. Optimization Objectives in WRSNs
  5. Optimization Approaches in WRSNs
  6. Models and Preliminaries
  7. Network Segments
  8. Wireless Power Transfer Model
  9. Wireless Charging Pile Model
  10. Movement and Energy Consumption Models of Mobile Charger
  11. Problem Formulation and Analysis
  12. Problem Statement
  13. Problem Analyses
  14. Algorithm Design
  15. MOMDP Formulation
  16. ...and 20 more sections

Key Result

Theorem 1

The MOP formulated in Eq. eq:formulation is NP-hard.

Figures (9)

  • Figure 1: The model of the considered MCA-WRSN system with the mobile charger, sensor nodes, and wireless charging pile. Green nodes denote normal sensor nodes, while red nodes denote emergency nodes requiring immediate charging.
  • Figure 2: Framework of EMODRL in MCA-WRSN. The framework begins with a warm-up stage to generate initial task population. The evolutionary stage iteratively performs task selection, LSTM-MPPO optimization (black dashed box), offspring generation, and Pareto archive updates through MLP-based prospective increment model and time-varying evaluation to obtain near-optimal policies.
  • Figure 3: LSTM-based policy network structure. LSTM cells process state $s_t$ through hidden states $h_t$, then fully connected layers output action $a_t$.
  • Figure 4: The results obtained by EMOPPO-TML. Subfigure (a) compares the Pareto policy distributions between EMOPPO-TML and baseline algorithms. Subfigures (b) and (c) display the node survival rate and energy usage efficiency over time.
  • Figure 5: Numerical results in terms of node survival rate and energy usage efficiency obtained by different baselines.
  • ...and 4 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof