Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hierarchical Online-Scheduling for Energy-Efficient Split Inference with Progressive Transmission

Zengzipeng Tang, Yuxuan Sun, Wei Chen, Jianwen Ding, Bo Ai, Yulin Shao

TL;DR

This work tackles deadline-sensitive, energy-constrained edge inference by addressing granularity mismatch and task heterogeneity in split DNN execution. It introduces ENACHI, a two-tier Lyapunov-based framework that couples a task-level surrogate-based optimization (partitioning, bandwidth, and a frame-level power budget) with a packet-level, uncertainty-aware progressive transmission inner loop that tracks the power reference and adaptively transmits feature maps. Theoretical guarantees (drift-plus-penalty bounds) accompany practical mechanisms like importance-aware packet selection and stopping criteria, yielding provable stability and near-optimality relative to offline benchmarks. Empirically, ENACHI on ImageNet with ResNet-50 demonstrates substantial gains in inference accuracy (up to 43.12%) and energy savings (up to 62.13%) under stringent deadlines and congested multi-user settings, indicating strong potential for scalable, energy-efficient edge AI deployments.

Abstract

Device-edge collaborative inference with Deep Neural Networks (DNNs) faces fundamental trade-offs among accuracy, latency and energy consumption. Current scheduling exhibits two drawbacks: a granularity mismatch between coarse, task-level decisions and fine-grained, packet-level channel dynamics, and insufficient awareness of per-task complexity. Consequently, scheduling solely at the task level leads to inefficient resource utilization. This paper proposes a novel ENergy-ACcuracy Hierarchical optimization framework for split Inference, named ENACHI, that jointly optimizes task- and packet-level scheduling to maximize accuracy under energy and delay constraints. A two-tier Lyapunov-based framework is developed for ENACHI, with a progressive transmission technique further integrated to enhance adaptivity. At the task level, an outer drift-plus-penalty loop makes online decisions for DNN partitioning and bandwidth allocation, and establishes a reference power budget to manage the long-term energy-accuracy trade-off. At the packet level, an uncertainty-aware progressive transmission mechanism is employed to adaptively manage per-sample task complexity. This is integrated with a nested inner control loop implementing a novel reference-tracking policy, which dynamically adjusts per-slot transmit power to adapt to fluctuating channel conditions. Experiments on ImageNet dataset demonstrate that ENACHI outperforms state-of-the-art benchmarks under varying deadlines and bandwidths, achieving a 43.12\% gain in inference accuracy with a 62.13\% reduction in energy consumption under stringent deadlines, and exhibits high scalability by maintaining stable energy consumption in congested multi-user scenarios.

Hierarchical Online-Scheduling for Energy-Efficient Split Inference with Progressive Transmission

TL;DR

This work tackles deadline-sensitive, energy-constrained edge inference by addressing granularity mismatch and task heterogeneity in split DNN execution. It introduces ENACHI, a two-tier Lyapunov-based framework that couples a task-level surrogate-based optimization (partitioning, bandwidth, and a frame-level power budget) with a packet-level, uncertainty-aware progressive transmission inner loop that tracks the power reference and adaptively transmits feature maps. Theoretical guarantees (drift-plus-penalty bounds) accompany practical mechanisms like importance-aware packet selection and stopping criteria, yielding provable stability and near-optimality relative to offline benchmarks. Empirically, ENACHI on ImageNet with ResNet-50 demonstrates substantial gains in inference accuracy (up to 43.12%) and energy savings (up to 62.13%) under stringent deadlines and congested multi-user settings, indicating strong potential for scalable, energy-efficient edge AI deployments.

Abstract

Device-edge collaborative inference with Deep Neural Networks (DNNs) faces fundamental trade-offs among accuracy, latency and energy consumption. Current scheduling exhibits two drawbacks: a granularity mismatch between coarse, task-level decisions and fine-grained, packet-level channel dynamics, and insufficient awareness of per-task complexity. Consequently, scheduling solely at the task level leads to inefficient resource utilization. This paper proposes a novel ENergy-ACcuracy Hierarchical optimization framework for split Inference, named ENACHI, that jointly optimizes task- and packet-level scheduling to maximize accuracy under energy and delay constraints. A two-tier Lyapunov-based framework is developed for ENACHI, with a progressive transmission technique further integrated to enhance adaptivity. At the task level, an outer drift-plus-penalty loop makes online decisions for DNN partitioning and bandwidth allocation, and establishes a reference power budget to manage the long-term energy-accuracy trade-off. At the packet level, an uncertainty-aware progressive transmission mechanism is employed to adaptively manage per-sample task complexity. This is integrated with a nested inner control loop implementing a novel reference-tracking policy, which dynamically adjusts per-slot transmit power to adapt to fluctuating channel conditions. Experiments on ImageNet dataset demonstrate that ENACHI outperforms state-of-the-art benchmarks under varying deadlines and bandwidths, achieving a 43.12\% gain in inference accuracy with a 62.13\% reduction in energy consumption under stringent deadlines, and exhibits high scalability by maintaining stable energy consumption in congested multi-user scenarios.
Paper Structure (27 sections, 4 theorems, 47 equations, 6 figures, 1 table, 2 algorithms)

This paper contains 27 sections, 4 theorems, 47 equations, 6 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. System Model
  3. Split Inference Model
  4. Task-aware Progressive Packet Transmission Model
  5. Batched Task Execution Model at the Edge
  6. Inference Task Accuracy Model
  7. Problem Formulation
  8. ENACHI: Energy-Accuracy Hierarchical Optimization Algorithm for Split Inference
  9. Problem Conversion
  10. Stage I — Task-Level Resource Scheduling via Surrogate Model
  11. Stage II — Packet-Level Reference-Tracking and Adaptation
  12. Task-Level Resource Scheduling via Surrogate Model
  13. Packet-Level Reference-Tracking and Adaptation
  14. Performance Analysis
  15. Experiments
  16. ...and 12 more sections

Key Result

Lemma 1

Using Lyapunov drift-plus-penalty framework, the long-term stochastic optimization problem $\mathcal{P}1$ can be transformed into the online, per-task optimization problem $\mathcal{P}1.1$:

Figures (6)

  • Figure 1: Illustration of the multi-user split inference system.
  • Figure 2: Illustration of the progressive packet transmission mechanism.
  • Figure 3: Illustration of the two-stage scheduling workflow.
  • Figure 4: The experimental and fitted curves of image classification task. L1 to L4 are the selected representative partition layers from shallow to deep.
  • Figure 5: Inference accuracy and energy consumption under different $V$.
  • ...and 1 more figures

Theorems & Definitions (8)

  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Theorem 1
  • proof