Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Inference Placement for Deep Learning across Edge and Cloud Platforms: A Multi-Objective Optimization Perspective and Future Directions

Zongshun Zhang, Ibrahim Matta

TL;DR

This survey addresses the challenge of running deep learning inferences across the device–edge–cloud continuum by formulating a multi‑objective optimization problem that jointly minimizes latency, monetary cost, and privacy loss. It introduces latency, cost, and privacy formulations and surveys a broad set of solutions, including dynamic partitioning, early exits, input/output and model compression, and privacy‑preserving adaptations. Through case studies, the work highlights the trade‑offs among the objectives and discusses open issues such as fine‑grained resource orchestration and defenses against model inversion and prompt inversion attacks. The findings aim to guide the design of future edge‑cloud MLaaS platforms that deliver low latency, strong privacy, and cost efficiency for DL inference tasks.

Abstract

Edge intelligent applications like VR/AR and language model based chatbots have become widespread with the rapid expansion of IoT and mobile devices. However, constrained edge devices often cannot serve the increasingly large and complex deep learning (DL) models. To mitigate these challenges, researchers have proposed optimizing and offloading partitions of DL models among user devices, edge servers, and the cloud. In this setting, users can take advantage of different services to support their intelligent applications. For example, edge resources offer low response latency. In contrast, cloud platforms provide low monetary cost computation resources for computation-intensive workloads. However, communication between DL model partitions can introduce transmission bottlenecks and pose risks of data leakage. Recent research aims to balance accuracy, computation delay, transmission delay, and privacy concerns. They address these issues with model compression, model distillation, transmission compression, and model architecture adaptations, including internal classifiers. This survey contextualizes the state-of-the-art model offloading methods and model adaptation techniques by studying their implication to a multi-objective optimization comprising inference latency, data privacy, and resource monetary cost.

Rethinking Inference Placement for Deep Learning across Edge and Cloud Platforms: A Multi-Objective Optimization Perspective and Future Directions

TL;DR

This survey addresses the challenge of running deep learning inferences across the device–edge–cloud continuum by formulating a multi‑objective optimization problem that jointly minimizes latency, monetary cost, and privacy loss. It introduces latency, cost, and privacy formulations and surveys a broad set of solutions, including dynamic partitioning, early exits, input/output and model compression, and privacy‑preserving adaptations. Through case studies, the work highlights the trade‑offs among the objectives and discusses open issues such as fine‑grained resource orchestration and defenses against model inversion and prompt inversion attacks. The findings aim to guide the design of future edge‑cloud MLaaS platforms that deliver low latency, strong privacy, and cost efficiency for DL inference tasks.

Abstract

Edge intelligent applications like VR/AR and language model based chatbots have become widespread with the rapid expansion of IoT and mobile devices. However, constrained edge devices often cannot serve the increasingly large and complex deep learning (DL) models. To mitigate these challenges, researchers have proposed optimizing and offloading partitions of DL models among user devices, edge servers, and the cloud. In this setting, users can take advantage of different services to support their intelligent applications. For example, edge resources offer low response latency. In contrast, cloud platforms provide low monetary cost computation resources for computation-intensive workloads. However, communication between DL model partitions can introduce transmission bottlenecks and pose risks of data leakage. Recent research aims to balance accuracy, computation delay, transmission delay, and privacy concerns. They address these issues with model compression, model distillation, transmission compression, and model architecture adaptations, including internal classifiers. This survey contextualizes the state-of-the-art model offloading methods and model adaptation techniques by studying their implication to a multi-objective optimization comprising inference latency, data privacy, and resource monetary cost.
Paper Structure (39 sections, 16 equations, 6 figures, 12 tables)

This paper contains 39 sections, 16 equations, 6 figures, 12 tables.

Table of Contents

  1. Introduction
  2. The Cost Model of Machine Learning-as-a-Service
  3. What are the limitations of existing MLaaS systems for ML Inference?
  4. Opportunities and Challenges of DL Task offloading during Inference
  5. Why offloading Deep Learning (DL) tasks?
  6. Challenges and Explorations in ML task offloading.
  7. Problem Definition
  8. DNN Offloading Challenges
  9. Latency
  10. Monetary Cost
  11. Privacy
  12. Problem Formulation
  13. Latency
  14. Latency Formulation
  15. Monetary Cost
  16. ...and 24 more sections

Figures (6)

  • Figure 1: A MLaaS System Offloads Service/Applications $s^{i}$ across the Continuum of Device, Edge and Cloud Resources Using Different Resource Orchestration Platforms.
  • Figure 2: Hidden variable sizes of VGG16 with CiFAR-10 and batch size of $16$.
  • Figure 3: Illustration of latency optimization problem.
  • Figure 4: Illustration of cost($) optimization problem.
  • Figure 5: Illustration of privacy optimization problem.
  • ...and 1 more figures