Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Hierarchical Optimization Framework Using Deep Reinforcement Learning for Task-Driven Bandwidth Allocation in 5G Teleoperation

Narges Golmohammadi, Madan Mohan Rayguru, Sabur Baidya

TL;DR

The paper tackles task-driven bandwidth allocation for 5G-enabled teleoperation by introducing a two-level hierarchy: a DRL-based upper level that allocates resources across URLLC and eMBB slices using a Lyapunov-prioritized objective, and a Razumikhin-based lower level that computes robust control gains to maintain stability under network delays. The framework leverages dual-queue modeling, network slicing, and a Lagrangian formulation to balance reliability, latency, and throughput while ensuring telerobotic control performance. Through simulations with realistic 5G settings, the authors show DRL convergence, reduced queue lengths, and improved resilience to varying channel conditions, sampling intervals, and Dexterity Index values, outperforming PF baselines in delay violation metrics. The study demonstrates a practical approach to joint network-control optimization for tactile Internet applications, enabling reliable, responsive teleoperation in dynamic wireless environments.

Abstract

The evolution of 5G wireless technology has revolutionized connectivity, enabling a diverse range of applications. Among these are critical use cases such as real time teleoperation, which demands ultra reliable low latency communications (URLLC) to ensure precise and uninterrupted control, and enhanced mobile broadband (eMBB) services, which cater to data-intensive applications requiring high throughput and bandwidth. In our scenario, there are two queues, one for eMBB users and one for URLLC users. In teleoperation tasks, control commands are received in the URLLC queue, where communication delays occur. The dynamic index (DI) controls the service rate, affecting the telerobotic (URLLC) queue. A separate queue models eMBB data traffic. Both queues are managed through network slicing and application delay constraints, leading to a unified Lagrangian-based Lyapunov optimization for efficient resource allocation. We propose a DRL based hierarchical optimization framework that consists of two levels. At the first level, network optimization dynamically allocates resources for eMBB and URLLC users using a Lagrangian functional and an actor critic network to balance competing objectives. At the second level, control optimization finetunes the best gains for robots, ensuring stability and responsiveness in network conditions. This hierarchical approach enhances both communication and control processes, ensuring efficient resource utilization and optimized performance across the network.

A Hierarchical Optimization Framework Using Deep Reinforcement Learning for Task-Driven Bandwidth Allocation in 5G Teleoperation

TL;DR

The paper tackles task-driven bandwidth allocation for 5G-enabled teleoperation by introducing a two-level hierarchy: a DRL-based upper level that allocates resources across URLLC and eMBB slices using a Lyapunov-prioritized objective, and a Razumikhin-based lower level that computes robust control gains to maintain stability under network delays. The framework leverages dual-queue modeling, network slicing, and a Lagrangian formulation to balance reliability, latency, and throughput while ensuring telerobotic control performance. Through simulations with realistic 5G settings, the authors show DRL convergence, reduced queue lengths, and improved resilience to varying channel conditions, sampling intervals, and Dexterity Index values, outperforming PF baselines in delay violation metrics. The study demonstrates a practical approach to joint network-control optimization for tactile Internet applications, enabling reliable, responsive teleoperation in dynamic wireless environments.

Abstract

The evolution of 5G wireless technology has revolutionized connectivity, enabling a diverse range of applications. Among these are critical use cases such as real time teleoperation, which demands ultra reliable low latency communications (URLLC) to ensure precise and uninterrupted control, and enhanced mobile broadband (eMBB) services, which cater to data-intensive applications requiring high throughput and bandwidth. In our scenario, there are two queues, one for eMBB users and one for URLLC users. In teleoperation tasks, control commands are received in the URLLC queue, where communication delays occur. The dynamic index (DI) controls the service rate, affecting the telerobotic (URLLC) queue. A separate queue models eMBB data traffic. Both queues are managed through network slicing and application delay constraints, leading to a unified Lagrangian-based Lyapunov optimization for efficient resource allocation. We propose a DRL based hierarchical optimization framework that consists of two levels. At the first level, network optimization dynamically allocates resources for eMBB and URLLC users using a Lagrangian functional and an actor critic network to balance competing objectives. At the second level, control optimization finetunes the best gains for robots, ensuring stability and responsiveness in network conditions. This hierarchical approach enhances both communication and control processes, ensuring efficient resource utilization and optimized performance across the network.

Paper Structure

This paper contains 20 sections, 18 equations, 8 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. System Model
  4. Application Scenario
  5. Wireless Communication Channel Model
  6. Network Queues
  7. Proposed Hierarchical Framework
  8. Upper Level Optimization Problem
  9. Lower Level Optimization
  10. Deep Reinforcement Learning (DRL) Framework
  11. Experiment and Results
  12. Simulation Setup
  13. Results
  14. DRL Performance
  15. Varying Network Conditions
  16. ...and 5 more sections

Figures (8)

  • Figure 1: Proposed Hierarchical Optimization Framework showing the components and dataflow
  • Figure 2: Reward for eMBB and URLLC Slices
  • Figure 3: Queue length of eMBB and URLLC users
  • Figure 4: CDF of eMBB and URLLC datarates with Varying $\sigma$ (Rayleigh Fading Scale) in the Wireless Network.
  • Figure 5: Avg. eMBB and URLLC Datarate and Queue Length for Varying $\sigma$ (Rayleigh Fading Scale) in the Wireless Network
  • ...and 3 more figures