Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AoI-Aware Task Offloading and Transmission Optimization for Industrial IoT Networks: A Branching Deep Reinforcement Learning Approach

Yuang Chen, Fengqian Guo, Chang Wu, Shuyi Liu, Hancheng Lu, Chang Wen Chen

TL;DR

This paper tackles the challenge of maintaining information freshness in IIoT by minimizing the long-term AoI in a multi-BS MEC setting. It introduces a Branching-D3QN for scalable, AoI-aware task offloading and proves convexity of the resource allocation subproblem to enable efficient joint optimization via alternating optimization. The proposed approach significantly accelerates learning (up to 75% faster convergence) and reduces the long-term AoI by at least 22% compared with state-of-the-art DRL and heuristic baselines. The work offers a scalable framework for real-time IIoT monitoring with practical impact on latency-sensitive industrial applications, and suggests extending to dynamic service migration in future work.

Abstract

In the Industrial Internet of Things (IIoT), the frequent transmission of large amounts of data over wireless networks should meet the stringent timeliness requirements. Particularly, the freshness of packet status updates has a significant impact on the system performance. In this paper, we propose an age-of-information (AoI)-aware multi-base station (BS) real-time monitoring framework to support extensive IIoT deployments. To meet the freshness requirements of IIoT, we formulate a joint task offloading and resource allocation optimization problem with the goal of minimizing long-term average AoI. Tackling the core challenges of combinatorial explosion in multi-BS decision spaces and the stochastic dynamics of IIoT systems is crucial, as these factors render traditional optimization methods intractable. Firstly, an innovative branching-based Dueling Double Deep Q-Network (Branching-D3QN) algorithm is proposed to effectively implement task offloading, which optimizes the convergence performance by reducing the action space complexity from exponential to linear levels. Then, an efficient optimization solution to resource allocation is proposed by proving the semi-definite property of the Hessian matrix of bandwidth and computation resources. Finally, we propose an iterative optimization algorithm for efficient joint task offloading and resource allocation to achieve optimal average AoI performance. Extensive simulations demonstrate that our proposed Branching-D3QN algorithm outperforms both state-of-the-art DRL methods and classical heuristics, achieving up to a 75% enhanced convergence speed and at least a 22% reduction in the long-term average AoI.

AoI-Aware Task Offloading and Transmission Optimization for Industrial IoT Networks: A Branching Deep Reinforcement Learning Approach

TL;DR

This paper tackles the challenge of maintaining information freshness in IIoT by minimizing the long-term AoI in a multi-BS MEC setting. It introduces a Branching-D3QN for scalable, AoI-aware task offloading and proves convexity of the resource allocation subproblem to enable efficient joint optimization via alternating optimization. The proposed approach significantly accelerates learning (up to 75% faster convergence) and reduces the long-term AoI by at least 22% compared with state-of-the-art DRL and heuristic baselines. The work offers a scalable framework for real-time IIoT monitoring with practical impact on latency-sensitive industrial applications, and suggests extending to dynamic service migration in future work.

Abstract

In the Industrial Internet of Things (IIoT), the frequent transmission of large amounts of data over wireless networks should meet the stringent timeliness requirements. Particularly, the freshness of packet status updates has a significant impact on the system performance. In this paper, we propose an age-of-information (AoI)-aware multi-base station (BS) real-time monitoring framework to support extensive IIoT deployments. To meet the freshness requirements of IIoT, we formulate a joint task offloading and resource allocation optimization problem with the goal of minimizing long-term average AoI. Tackling the core challenges of combinatorial explosion in multi-BS decision spaces and the stochastic dynamics of IIoT systems is crucial, as these factors render traditional optimization methods intractable. Firstly, an innovative branching-based Dueling Double Deep Q-Network (Branching-D3QN) algorithm is proposed to effectively implement task offloading, which optimizes the convergence performance by reducing the action space complexity from exponential to linear levels. Then, an efficient optimization solution to resource allocation is proposed by proving the semi-definite property of the Hessian matrix of bandwidth and computation resources. Finally, we propose an iterative optimization algorithm for efficient joint task offloading and resource allocation to achieve optimal average AoI performance. Extensive simulations demonstrate that our proposed Branching-D3QN algorithm outperforms both state-of-the-art DRL methods and classical heuristics, achieving up to a 75% enhanced convergence speed and at least a 22% reduction in the long-term average AoI.
Paper Structure (20 sections, 1 theorem, 26 equations, 13 figures, 1 table, 2 algorithms)

This paper contains 20 sections, 1 theorem, 26 equations, 13 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Motivations and Challenges
  3. Main Contributions
  4. RELATED WORKS
  5. Research on AoI-based Timeliness for Task offloading and Resource Allocation
  6. Studies on DRL-driven task offloading and resource allocation schemes in MEC Networks
  7. Limitations of these Existing Research Works
  8. System Model
  9. Problem Formulation
  10. Algorithm Designs and Solutions
  11. The Solution for the Task Offloading Problem
  12. The Solution for Resource Allocation Problem
  13. Joint Task Offloading and Resource Allocation Scheme
  14. Performance Evaluation
  15. Experimental Parameter Settings
  16. ...and 5 more sections

Key Result

Theorem 1

The objective function of $\mathcal{P}4$ is convex regarding the resource allocation variables ${B_{i,j}(t), f_{i,j}(t)}_{\forall i \in \mathcal{N}, j \in \mathcal{M}}$, and its all constraints are convex. Thus, $\mathcal{P}4$ is a strictly convex optimization problem.

Figures (13)

  • Figure 1: The proposed AoI-aware multi-BS real-time monitoring system for IIoT access.
  • Figure 2: The AoI changes of the proposed multi-BS-MEC real-time monitoring system for IIoT access.
  • Figure 3: The basic framework of DQN framework mnih2015human.
  • Figure 4: The D3QN framework designed for our considered task offloading subproblem.
  • Figure 5: The framework of the Branching-D3QN algorithm.
  • ...and 8 more figures

Theorems & Definitions (2)

  • Theorem 1
  • proof