Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels

Hao Wu, Shengtian Yang, Huiguo Gao, Diao Wang, Jun Chen, Guanding Yu

TL;DR

This work addresses online power control for energy-harvesting wireless links over fading channels by deriving a linear-policy approximation to the Bellman relative-value function and introducing two clipped affine policies: optimistic and robust. These policies yield a battery-limited directional waterfilling interpretation and enable a domain-knowledge-enhanced RL scheme for tuning a small set of parameters, with extensions to one-step energy and/or channel lookahead. Empirical results show the robust clipped affine policy, combined with RL, achieves less than 2% performance loss relative to the optimal policy across varied scenarios, outperforming existing methods, while maintaining low computational complexity. The approach provides a practical, high-performing building block for EH systems and highlights how problem structure can be exploited to design near-optimal online controls with modest training requirements.

Abstract

This paper investigates online power control for point-to-point energy harvesting communications over wireless fading channels. A linear-policy-based approximation is derived for the relative-value function in the Bellman equation of the power control problem. This approximation leads to two fundamental power control policies: optimistic and robust clipped affine policies, both taking the form of a clipped affine function of the battery level and the reciprocal of channel signal-to-noise ratio coefficient. They are essentially battery-limited weighted directional waterfilling policies operating between adjacent time slots. By leveraging the relative-value approximation and derived policies, a domain-knowledge-enhanced reinforcement learning (RL) algorithm is proposed for online power control. The proposed approach is further extended to scenarios with energy and/or channel lookahead. Comprehensive simulation results demonstrate that the proposed methods achieve a good balance between computational complexity and optimality. In particular, the robust clipped affine policy (combined with RL, using at most five parameters) outperforms all existing approaches across various scenarios, with less than 2\% performance loss relative to the optimal policy.

Clipped Affine Policy: Low-Complexity Near-Optimal Online Power Control for Energy Harvesting Communications over Fading Channels

TL;DR

This work addresses online power control for energy-harvesting wireless links over fading channels by deriving a linear-policy approximation to the Bellman relative-value function and introducing two clipped affine policies: optimistic and robust. These policies yield a battery-limited directional waterfilling interpretation and enable a domain-knowledge-enhanced RL scheme for tuning a small set of parameters, with extensions to one-step energy and/or channel lookahead. Empirical results show the robust clipped affine policy, combined with RL, achieves less than 2% performance loss relative to the optimal policy across varied scenarios, outperforming existing methods, while maintaining low computational complexity. The approach provides a practical, high-performing building block for EH systems and highlights how problem structure can be exploited to design near-optimal online controls with modest training requirements.

Abstract

This paper investigates online power control for point-to-point energy harvesting communications over wireless fading channels. A linear-policy-based approximation is derived for the relative-value function in the Bellman equation of the power control problem. This approximation leads to two fundamental power control policies: optimistic and robust clipped affine policies, both taking the form of a clipped affine function of the battery level and the reciprocal of channel signal-to-noise ratio coefficient. They are essentially battery-limited weighted directional waterfilling policies operating between adjacent time slots. By leveraging the relative-value approximation and derived policies, a domain-knowledge-enhanced reinforcement learning (RL) algorithm is proposed for online power control. The proposed approach is further extended to scenarios with energy and/or channel lookahead. Comprehensive simulation results demonstrate that the proposed methods achieve a good balance between computational complexity and optimality. In particular, the robust clipped affine policy (combined with RL, using at most five parameters) outperforms all existing approaches across various scenarios, with less than 2\% performance loss relative to the optimal policy.
Paper Structure (12 sections, 6 theorems, 65 equations, 5 figures, 7 tables, 4 algorithms)

This paper contains 12 sections, 6 theorems, 65 equations, 5 figures, 7 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Clipped Affine Policy
  4. Online Power Control Based on Clipped Affine Policies and Reinforcement Learning
  5. Power Control with One-Step Lookahead
  6. One-Step Energy Lookahead
  7. One-Step Channel Lookahead
  8. One-Step Energy-Channel Lookahead
  9. Simulation Results
  10. Conclusion
  11. Proofs of Theorems \ref{['th:onepoint_bellman2_solution']}--\ref{['th:onepoint_bellman2_solution2']}
  12. Proofs of Theorems \ref{['th:optimistic_clipped_affine_policy']} and \ref{['th:robust_clipped_affine_policy']}

Key Result

Theorem 1

If there exist a constant $g$ and a bounded function $h: [0,c] \times [0,+\infty) \to \mathbb{R}$ such that where $(E,\Gamma) \buildrel \text{d} \over = (E_t,\Gamma_{t+1})$, then $g^* = g$. Furthermore, if there exists a stationary policy $\sigma$ such that then $\mathcal{G}(\sigma) = g^*$.

Figures (5)

  • Figure 1: A discrete-time energy harvesting wireless communication system.
  • Figure 2: An illustration of Algorithm \ref{['alg:online_power_control']} (for $\sigma= \sigma_{\text{rca} }$).
  • Figure 3: Performance comparison under Bernoullli energy arrivals.
  • Figure 4: Performance comparison under exponential energy arrivals.
  • Figure 5: Performance comparison under uniform energy arrivals.

Theorems & Definitions (7)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 5: Optimistic Clipped Affine Policy
  • Theorem 6: Robust Clipped Affine Policy
  • Remark 1