Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Reinforcement Learning Based Universal Sequence Design for Polar Codes

David Kin Wai Ho, Arman Fazeli, Mohamad M. Mansour, Louay M. A. Jalloul

TL;DR

This work addresses scalable universal design of Polar code sequences across varying block lengths and payloads by framing the problem as a reinforcement-learning task. It introduces PPO-based learning with universal partial order constraints, lower-$N$ embedding, iterative lookahead, and joint multi-configuration optimization to enable training up to $N_{max}=2048$ and achieve competitive performance relative to the 5G NR sequence, with up to $0.2$ dB gains at $N=2048$ over beta-expansion. The approach demonstrates practical viability for 6G standardization by balancing performance, scalability, and cross-configuration knowledge transfer, and it provides open-source code for reproducibility. The key contribution is showing that physically informed constraints (UPO) combined with modern deep RL can reliably design universal reliability sequences across a wide range of code lengths and decoding settings, offering a path toward data-driven, scalable channel-code construction.

Abstract

To advance Polar code design for 6G applications, we develop a reinforcement learning-based universal sequence design framework that is extensible and adaptable to diverse channel conditions and decoding strategies. Crucially, our method scales to code lengths up to $2048$, making it suitable for use in standardization. Across all $(N,K)$ configurations supported in 5G, our approach achieves competitive performance relative to the NR sequence adopted in 5G and yields up to a 0.2 dB gain over the beta-expansion baseline at $N=2048$. We further highlight the key elements that enabled learning at scale: (i) incorporation of physical law constrained learning grounded in the universal partial order property of Polar codes, (ii) exploitation of the weak long term influence of decisions to limit lookahead evaluation, and (iii) joint multi-configuration optimization to increase learning efficiency.

A Reinforcement Learning Based Universal Sequence Design for Polar Codes

TL;DR

This work addresses scalable universal design of Polar code sequences across varying block lengths and payloads by framing the problem as a reinforcement-learning task. It introduces PPO-based learning with universal partial order constraints, lower- embedding, iterative lookahead, and joint multi-configuration optimization to enable training up to and achieve competitive performance relative to the 5G NR sequence, with up to dB gains at over beta-expansion. The approach demonstrates practical viability for 6G standardization by balancing performance, scalability, and cross-configuration knowledge transfer, and it provides open-source code for reproducibility. The key contribution is showing that physically informed constraints (UPO) combined with modern deep RL can reliably design universal reliability sequences across a wide range of code lengths and decoding settings, offering a path toward data-driven, scalable channel-code construction.

Abstract

To advance Polar code design for 6G applications, we develop a reinforcement learning-based universal sequence design framework that is extensible and adaptable to diverse channel conditions and decoding strategies. Crucially, our method scales to code lengths up to , making it suitable for use in standardization. Across all configurations supported in 5G, our approach achieves competitive performance relative to the NR sequence adopted in 5G and yields up to a 0.2 dB gain over the beta-expansion baseline at . We further highlight the key elements that enabled learning at scale: (i) incorporation of physical law constrained learning grounded in the universal partial order property of Polar codes, (ii) exploitation of the weak long term influence of decisions to limit lookahead evaluation, and (iii) joint multi-configuration optimization to increase learning efficiency.
Paper Structure (31 sections, 3 equations, 10 figures, 2 algorithms)

This paper contains 31 sections, 3 equations, 10 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Polar Codes in 5G NR and beyond
  3. Adoption of universal reliability sequence
  4. Reinforcement learning in code construction
  5. Universal Partial Order
  6. Physical law constrained machine learning
  7. Contributions
  8. Problem Definition
  9. Proposed Method
  10. High level description
  11. Action space constraint management
  12. Universal partial order
  13. Benefit of joint multi configuration optimization
  14. Restrict search with limited lookahead
  15. Deep reinforcement learning
  16. ...and 16 more sections

Figures (10)

  • Figure 1: UPO lattice with the current context in yellow, the immediate neighborhood in green and the disallowed neighbor in red. Our proposed relaxation will include the red node as permissible action in the current context, as this is the immediate neighbor of node 27.
  • Figure 2: Performance impact of iterative learning in joint multi-configuration optimization.
  • Figure 3: Required SNR characterization for baseline beta-expansion (PW) sequence used to define evaluation anchors.
  • Figure 4: Polar code performance under CRC-aided successive cancellation list decoding over an AWGN channel for the beta expansion (PW), NR and RL-learned reliability sequences.
  • Figure 5: Comparison of RL-learned sequence vs beta expansion at $N=2048$, first part.
  • ...and 5 more figures

Theorems & Definitions (5)

  • Definition 3.1
  • Remark 3.2
  • Remark 3.3
  • Remark 4.1
  • Remark 4.2