Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Practical Policy Distillation for Reinforcement Learning in Radio Access Networks

Sara Khosravi, Burak Demirel, Linghui Zhou, Javier Rasines, Pablo Soldati

TL;DR

The paper addresses deploying RL-based link adaptation in radio access networks under tight hardware and latency constraints. It proposes offline policy distillation with domain randomization to compress a large teacher policy into lightweight students, yielding two strategies: single-policy distillation and multi-policy distillation. Experiments in a 5G-compliant simulator show substantial model compression (on the order of large reductions) while preserving near-teacher performance across diverse scenarios, with manageable degradation in throughput and BLER. This work demonstrates a practical, scalable path to embedding AI-driven LA in legacy baseband hardware, enabling robust generalization across heterogeneous RAN deployments.

Abstract

Adopting artificial intelligence (AI) in radio access networks (RANs) presents several challenges, including limited availability of link-level measurements (e.g., CQI reports), stringent real-time processing constraints (e.g., sub-1 ms per TTI), and network heterogeneity (different spectrum bands, cell types, and vendor equipment). A critical yet often overlooked barrier lies in the computational and memory limitations of RAN baseband hardware, particularly in legacy 4th Generation (4G) systems, which typically lack on-chip neural accelerators. As a result, only lightweight AI models (under 1 Mb and sub-100~μs inference time) can be effectively deployed, limiting both their performance and applicability. However, achieving strong generalization across diverse network conditions often requires large-scale models with substantial resource demands. To address this trade-off, this paper investigates policy distillation in the context of a reinforcement learning-based link adaptation task. We explore two strategies: single-policy distillation, where a scenario-agnostic teacher model is compressed into one generalized student model; and multi-policy distillation, where multiple scenario-specific teachers are consolidated into a single generalist student. Experimental evaluations in a high-fidelity, 5th Generation (5G)-compliant simulator demonstrate that both strategies produce compact student models that preserve the teachers' generalization capabilities while complying with the computational and memory limitations of existing RAN hardware.

Practical Policy Distillation for Reinforcement Learning in Radio Access Networks

TL;DR

The paper addresses deploying RL-based link adaptation in radio access networks under tight hardware and latency constraints. It proposes offline policy distillation with domain randomization to compress a large teacher policy into lightweight students, yielding two strategies: single-policy distillation and multi-policy distillation. Experiments in a 5G-compliant simulator show substantial model compression (on the order of large reductions) while preserving near-teacher performance across diverse scenarios, with manageable degradation in throughput and BLER. This work demonstrates a practical, scalable path to embedding AI-driven LA in legacy baseband hardware, enabling robust generalization across heterogeneous RAN deployments.

Abstract

Adopting artificial intelligence (AI) in radio access networks (RANs) presents several challenges, including limited availability of link-level measurements (e.g., CQI reports), stringent real-time processing constraints (e.g., sub-1 ms per TTI), and network heterogeneity (different spectrum bands, cell types, and vendor equipment). A critical yet often overlooked barrier lies in the computational and memory limitations of RAN baseband hardware, particularly in legacy 4th Generation (4G) systems, which typically lack on-chip neural accelerators. As a result, only lightweight AI models (under 1 Mb and sub-100~μs inference time) can be effectively deployed, limiting both their performance and applicability. However, achieving strong generalization across diverse network conditions often requires large-scale models with substantial resource demands. To address this trade-off, this paper investigates policy distillation in the context of a reinforcement learning-based link adaptation task. We explore two strategies: single-policy distillation, where a scenario-agnostic teacher model is compressed into one generalized student model; and multi-policy distillation, where multiple scenario-specific teachers are consolidated into a single generalist student. Experimental evaluations in a high-fidelity, 5th Generation (5G)-compliant simulator demonstrate that both strategies produce compact student models that preserve the teachers' generalization capabilities while complying with the computational and memory limitations of existing RAN hardware.

Paper Structure

This paper contains 14 sections, 3 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. The Hidden Challenges of AI in RAN
  3. Problem formulation
  4. RL-based Link Adaptation
  5. Policy Distillation
  6. Offline Policy Distillation
  7. Generalization via Single-Policy Distillation
  8. Generalization via Multi-Policy Distillation
  9. Numerical examples
  10. Training Setup
  11. Testing Setup
  12. Generalization via Single-Policy Distillation
  13. Generalization via Multi-Policy Distillation
  14. Conclusions

Figures (3)

  • Figure 1: Example of MDP episode definition for LA.
  • Figure 2: Comparison of the average throughput achieved by each student relative to the teacher in the MIMO, mMIMO and SCSU benchmark scenarios.
  • Figure 3: Comparison of the teacher and students policies across the three benchmark scenarios in terms of: CDF of the UE throughput (a)-(c); and actions distribution, represented by the PDF of selected MCS values (d)-(f).