Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ECORE: Energy-Conscious Optimized Routing for Deep Learning Models at the Edge

Daghash K. Alqahtani, Maria A. Rodriguez, Muhammad Aamir Cheema, Hamid Rezatofighi, Adel N. Toosi

TL;DR

ECORE addresses energy-constrained edge vision by introducing context-aware routing that dynamically directs image frames to the most suitable heterogeneous edge device–model pair. It uses lightweight object-count estimators at a central gateway and a greedy routing rule constrained by a tolerance δmAP, formalized as the feasible set ${\mathcal{F}}=\{ i\in \mathcal{M}_G \mid \text{mAP}_i \ge \text{mAP}_{\max}-\delta_{\text{mAP}}\}$ with $i^*=\arg\min_{i\in\mathcal{F}} e_i$, and provides proofs of optimality under these conditions. The approach introduces three object-count estimation methods—Edge Detection via Canny, an SSD-based gateway front-end, and an Output-Based reuse mechanism—to keep routing lightweight. Empirical evaluation on COCO, a balanced dataset, and a pedestrian video demonstrates notable energy and latency reductions (up to 35% and 49%, respectively) with only about a 2% drop in mAP compared with accuracy-focused baselines, validating the practical impact of content-aware routing for energy-efficient edge AI.

Abstract

Edge computing enables data processing closer to the source, significantly reducing latency, an essential requirement for real-time vision-based analytics such as object detection in surveillance and smart city environments. However, these tasks place substantial demands on resource-constrained edge devices, making the joint optimization of energy consumption and detection accuracy critical. To address this challenge, we propose ECORE, a framework that integrates multiple dynamic routing strategies, including a novel estimation-based techniques and an innovative greedy selection algorithm, to direct image processing requests to the most suitable edge device-model pair. ECORE dynamically balances energy efficiency and detection performance based on object characteristics. We evaluate our framework through extensive experiments on real-world datasets, comparing against widely used baseline techniques. The evaluation leverages established object detection models (YOLO, SSD, EfficientDet) and diverse edge platforms, including Jetson Orin Nano, Raspberry Pi 4 and 5, and TPU accelerators. Results demonstrate that our proposed context-aware routing strategies can reduce energy consumption and latency by 35% and 49%, respectively, while incurring only a 2% loss in detection accuracy compared to accuracy-centric methods.

ECORE: Energy-Conscious Optimized Routing for Deep Learning Models at the Edge

TL;DR

ECORE addresses energy-constrained edge vision by introducing context-aware routing that dynamically directs image frames to the most suitable heterogeneous edge device–model pair. It uses lightweight object-count estimators at a central gateway and a greedy routing rule constrained by a tolerance δmAP, formalized as the feasible set with , and provides proofs of optimality under these conditions. The approach introduces three object-count estimation methods—Edge Detection via Canny, an SSD-based gateway front-end, and an Output-Based reuse mechanism—to keep routing lightweight. Empirical evaluation on COCO, a balanced dataset, and a pedestrian video demonstrates notable energy and latency reductions (up to 35% and 49%, respectively) with only about a 2% drop in mAP compared with accuracy-focused baselines, validating the practical impact of content-aware routing for energy-efficient edge AI.

Abstract

Edge computing enables data processing closer to the source, significantly reducing latency, an essential requirement for real-time vision-based analytics such as object detection in surveillance and smart city environments. However, these tasks place substantial demands on resource-constrained edge devices, making the joint optimization of energy consumption and detection accuracy critical. To address this challenge, we propose ECORE, a framework that integrates multiple dynamic routing strategies, including a novel estimation-based techniques and an innovative greedy selection algorithm, to direct image processing requests to the most suitable edge device-model pair. ECORE dynamically balances energy efficiency and detection performance based on object characteristics. We evaluate our framework through extensive experiments on real-world datasets, comparing against widely used baseline techniques. The evaluation leverages established object detection models (YOLO, SSD, EfficientDet) and diverse edge platforms, including Jetson Orin Nano, Raspberry Pi 4 and 5, and TPU accelerators. Results demonstrate that our proposed context-aware routing strategies can reduce energy consumption and latency by 35% and 49%, respectively, while incurring only a 2% loss in detection accuracy compared to accuracy-centric methods.

Paper Structure

This paper contains 22 sections, 1 theorem, 7 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation
  3. System Design and Implementation
  4. Problem Formulation and Greedy Approach
  5. Optimality of the Routing Algorithm
  6. Proposed object count estimation approaches
  7. Performance Evaluation
  8. Experimental Setup
  9. Dataset
  10. Testbed
  11. Experimental Procedure
  12. Experimental Results
  13. Full COCO Dataset Results:
  14. Balanced Sorted Dataset Results
  15. Video Dataset Results
  16. ...and 7 more sections

Key Result

Theorem 1

Let $\mathcal{M}_G$ be the subset of profiled model-device pairs matching the estimated object count group $G$, and let $\delta_{\text{mAP}}$ be a fixed accuracy tolerance. Then, the greedy selection of the model-device pair with the lowest energy consumption from the filtered feasible set yields an optimal solution to the routing problem under the specified constraints.

Figures (9)

  • Figure 1: Contrasting pedestrians scenarios: (a) High density scene; (b) Low density scene.
  • Figure 2: Energy Consumption and Accuracy for different group of images across object detection models.
  • Figure 3: System architecture presentation.
  • Figure 4: Distribution of Object Counts / Image in COCO dataset.
  • Figure 5: Pareto frontiers comparing object detection models across various edge devices.
  • ...and 4 more figures

Theorems & Definitions (2)

  • Theorem 1: Optimality of the Routing Algorithm
  • proof