Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Synergy: Towards On-Body AI via Tiny AI Accelerator Collaboration on Wearables

Taesik Gong, Si Young Jang, Utku Günay Acer, Fahim Kawsar, Chulhong Min

TL;DR

Synergy addresses the challenge of running multiple on-body AI apps concurrently on wearable devices equipped with tiny AI accelerators. It introduces a device-agnostic programming interface, holistic collaboration planning, a clock-cycle based latency model, online throughput estimation, and adaptive task parallelization to maximize system throughput while reducing latency and energy. The system partitions model layers across distributed accelerators and coordinates sensing, model inference, and interaction tasks across wearables, avoiding offloading to smartphones. Evaluations on MAX78000/MAX78002 demonstrate substantial throughput gains (average 23×), with meaningful latency reductions and modest energy savings, underscoring the practicality of on-body AI collaboration.

Abstract

The advent of tiny artificial intelligence (AI) accelerators enables AI to run at the extreme edge, offering reduced latency, lower power cost, and improved privacy. When integrated into wearable devices, these accelerators open exciting opportunities, allowing various AI apps to run directly on the body. We present Synergy that provides AI apps with best-effort performance via system-driven holistic collaboration over AI accelerator-equipped wearables. To achieve this, Synergy provides device-agnostic programming interfaces to AI apps, giving the system visibility and controllability over the app's resource use. Then, Synergy maximizes the inference throughput of concurrent AI models by creating various execution plans for each app considering AI accelerator availability and intelligently selecting the best set of execution plans. Synergy further improves throughput by leveraging parallelization opportunities over multiple computation units. Our evaluations with 7 baselines and 8 models demonstrate that, on average, Synergy achieves a 23.0 times improvement in throughput, while reducing latency by 73.9% and power consumption by 15.8%, compared to the baselines.

Synergy: Towards On-Body AI via Tiny AI Accelerator Collaboration on Wearables

TL;DR

Synergy addresses the challenge of running multiple on-body AI apps concurrently on wearable devices equipped with tiny AI accelerators. It introduces a device-agnostic programming interface, holistic collaboration planning, a clock-cycle based latency model, online throughput estimation, and adaptive task parallelization to maximize system throughput while reducing latency and energy. The system partitions model layers across distributed accelerators and coordinates sensing, model inference, and interaction tasks across wearables, avoiding offloading to smartphones. Evaluations on MAX78000/MAX78002 demonstrate substantial throughput gains (average 23×), with meaningful latency reductions and modest energy savings, underscoring the practicality of on-body AI collaboration.

Abstract

The advent of tiny artificial intelligence (AI) accelerators enables AI to run at the extreme edge, offering reduced latency, lower power cost, and improved privacy. When integrated into wearable devices, these accelerators open exciting opportunities, allowing various AI apps to run directly on the body. We present Synergy that provides AI apps with best-effort performance via system-driven holistic collaboration over AI accelerator-equipped wearables. To achieve this, Synergy provides device-agnostic programming interfaces to AI apps, giving the system visibility and controllability over the app's resource use. Then, Synergy maximizes the inference throughput of concurrent AI models by creating various execution plans for each app considering AI accelerator availability and intelligently selecting the best set of execution plans. Synergy further improves throughput by leveraging parallelization opportunities over multiple computation units. Our evaluations with 7 baselines and 8 models demonstrate that, on average, Synergy achieves a 23.0 times improvement in throughput, while reducing latency by 73.9% and power consumption by 15.8%, compared to the baselines.
Paper Structure (36 sections, 3 equations, 19 figures, 3 tables)

This paper contains 36 sections, 3 equations, 19 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background & Motivation
  3. Tiny AI Accelerators
  4. AI Accelerator Collaboration
  5. Why not offloading?
  6. Synergy Design
  7. Limitations of Existing Approaches
  8. Our Approach: AI Accelerator Collaboration with a Holistic View
  9. System Scope
  10. Synergy Runtime Technical Components
  11. Overview
  12. Summary of Technical Novelties
  13. Operational Flow
  14. Device-Agnostic Programming Interface
  15. Holistic Collaboration Planning
  16. ...and 21 more sections

Figures (19)

  • Figure 1: Wearable computing powered by tiny AI accelerators, simultaneously running on-body AI apps.
  • Figure 2: Performance comparison between AI accelerator (MAX78000) and MCUs (MAX32650 and STM32F7).
  • Figure 3: Comparison between (a) AI accelerator collaboration via Synergy and (b) phone offloading.
  • Figure 4: Comparison of Synergy and phone offloading.
  • Figure 5: Comparison of decision approaches.
  • ...and 14 more figures