Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Scene-aware Models Adaptation Scheme for Cross-scene Online Inference on Mobile Devices

Yunzhe Li, Hongzi Zhu, Zhuohong Deng, Yunlong Cheng, Zimu Zheng, Liang Zhang, Shan Chang, Minyi Guo

TL;DR

Anole tackles online mobile inference under cross-scene distribution drift by building a portfolio of compact, scene-specific DNNs and a decision model to select the best fit for each test sample. It introduces offline scene profiling to create model-friendly partitions via semantic and feature similarity, and online inference with a cache-based deployment strategy. Extensive experiments on UAV-driven datasets show Anole outperforms a single large DNN in accuracy, latency, and energy, including hard unseen scenes. The approach enables robust, cloud-free inference on resource-constrained mobile devices in dynamic environments.

Abstract

Emerging Artificial Intelligence of Things (AIoT) applications desire online prediction using deep neural network (DNN) models on mobile devices. However, due to the movement of devices, unfamiliar test samples constantly appear, significantly affecting the prediction accuracy of a pre-trained DNN. In addition, unstable network connection calls for local model inference. In this paper, we propose a light-weight scheme, called Anole, to cope with the local DNN model inference on mobile devices. The core idea of Anole is to first establish an army of compact DNN models, and then adaptively select the model fitting the current test sample best for online inference. The key is to automatically identify model-friendly scenes for training scene-specific DNN models. To this end, we design a weakly-supervised scene representation learning algorithm by combining both human heuristics and feature similarity in separating scenes. Moreover, we further train a model classifier to predict the best-fit scene-specific DNN model for each test sample. We implement Anole on different types of mobile devices and conduct extensive trace-driven and real-world experiments based on unmanned aerial vehicles (UAVs). The results demonstrate that Anole outwits the method of using a versatile large DNN in terms of prediction accuracy (4.5% higher), response time (33.1% faster) and power consumption (45.1% lower).

A Scene-aware Models Adaptation Scheme for Cross-scene Online Inference on Mobile Devices

TL;DR

Anole tackles online mobile inference under cross-scene distribution drift by building a portfolio of compact, scene-specific DNNs and a decision model to select the best fit for each test sample. It introduces offline scene profiling to create model-friendly partitions via semantic and feature similarity, and online inference with a cache-based deployment strategy. Extensive experiments on UAV-driven datasets show Anole outperforms a single large DNN in accuracy, latency, and energy, including hard unseen scenes. The approach enables robust, cloud-free inference on resource-constrained mobile devices in dynamic environments.

Abstract

Emerging Artificial Intelligence of Things (AIoT) applications desire online prediction using deep neural network (DNN) models on mobile devices. However, due to the movement of devices, unfamiliar test samples constantly appear, significantly affecting the prediction accuracy of a pre-trained DNN. In addition, unstable network connection calls for local model inference. In this paper, we propose a light-weight scheme, called Anole, to cope with the local DNN model inference on mobile devices. The core idea of Anole is to first establish an army of compact DNN models, and then adaptively select the model fitting the current test sample best for online inference. The key is to automatically identify model-friendly scenes for training scene-specific DNN models. To this end, we design a weakly-supervised scene representation learning algorithm by combining both human heuristics and feature similarity in separating scenes. Moreover, we further train a model classifier to predict the best-fit scene-specific DNN model for each test sample. We implement Anole on different types of mobile devices and conduct extensive trace-driven and real-world experiments based on unmanned aerial vehicles (UAVs). The results demonstrate that Anole outwits the method of using a versatile large DNN in terms of prediction accuracy (4.5% higher), response time (33.1% faster) and power consumption (45.1% lower).
Paper Structure (45 sections, 1 theorem, 1 equation, 13 figures, 6 tables, 2 algorithms)

This paper contains 45 sections, 1 theorem, 1 equation, 13 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. System Model
  4. Problem Formulation
  5. Empirical Study
  6. Driving Video Datasets
  7. Mobile Inference Accuracy Analysis
  8. Overview of Anole
  9. Offline Scene Profiling
  10. Training Compressed Models
  11. Training Dataset Segmentation
  12. Compressed Model Training
  13. Adaptive Scene Sampling
  14. Training Decision Model
  15. Online Model Inference
  16. ...and 30 more sections

Key Result

Proposition 1

Though $\mathcal{M}_i$ is trained on $\Gamma_i$, not all data samples in $\Gamma_i$ necessarily belong to $\Psi_i$, i.e., $\Gamma_i \not\subset \Psi_i$.

Figures (13)

  • Figure 1: Illustration of the online mobile inference problem, where data distributions characterized by statistical models (depicted as dashed disks) are implicit and not easy to understand.
  • Figure 2: The dataset of 64 randomly selected driving video clips demonstrates a large diversity in terms of image light conditions and foreground object distributions.
  • Figure 3: A best-fit compressed model can achieve comparable prediction accuracy provided by a large DNN, whereas compressed DNNs trained using existing scene partitioning methods fail to perform well on their corresponding training dataset.
  • Figure 4: System architecture of Anole, which consists of the offline scene profiling on cloud servers and the online model inference on mobile devices. Communication between both parts is carried out offline.
  • Figure 5: (a) An example of compressed models being unevenly sampled with random sampling; (b) our adaptive sampling algorithm can mitigate the unbalanced sampling problem.
  • ...and 8 more figures

Theorems & Definitions (1)

  • Proposition 1