Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Recommending Pre-Trained Models for IoT Devices

Parth V. Patil, Wenxin Jiang, Huiyun Peng, Daniel Lugo, Kelechi G. Kalu, Josh LeBlanc, Lawrence Smith, Hyeonwoo Heo, Nathanael Aou, James C. Davis

TL;DR

This work addresses the gap that IoT hardware constraints are underrepresented in pre-trained model (PTM) recommendations. It proposes hardware-aware extensions to the Model Spider framework (Model Spider Fusion and Model Spider Shadow) to jointly consider task relevance and hardware feasibility, complemented by a data collection and Copeland-based ranking strategy. By outlining a concrete dataset and evaluation methodology, the paper lays groundwork for ground-truth IoT device rankings and scalable, energy-aware PTM deployment. The approach promises more reliable, resource-conscious PTM selection for IoT applications, and it sketches a broader research agenda to extend to distributed DL systems and more complex model reuse scenarios.

Abstract

The availability of pre-trained models (PTMs) has enabled faster deployment of machine learning across applications by reducing the need for extensive training. Techniques like quantization and distillation have further expanded PTM applicability to resource-constrained IoT hardware. Given the many PTM options for any given task, engineers often find it too costly to evaluate each model's suitability. Approaches such as LogME, LEEP, and ModelSpider help streamline model selection by estimating task relevance without exhaustive tuning. However, these methods largely leave hardware constraints as future work-a significant limitation in IoT settings. In this paper, we identify the limitations of current model recommendation approaches regarding hardware constraints and introduce a novel, hardware-aware method for PTM selection. We also propose a research agenda to guide the development of effective, hardware-conscious model recommendation systems for IoT applications.

Recommending Pre-Trained Models for IoT Devices

TL;DR

This work addresses the gap that IoT hardware constraints are underrepresented in pre-trained model (PTM) recommendations. It proposes hardware-aware extensions to the Model Spider framework (Model Spider Fusion and Model Spider Shadow) to jointly consider task relevance and hardware feasibility, complemented by a data collection and Copeland-based ranking strategy. By outlining a concrete dataset and evaluation methodology, the paper lays groundwork for ground-truth IoT device rankings and scalable, energy-aware PTM deployment. The approach promises more reliable, resource-conscious PTM selection for IoT applications, and it sketches a broader research agenda to extend to distributed DL systems and more complex model reuse scenarios.

Abstract

The availability of pre-trained models (PTMs) has enabled faster deployment of machine learning across applications by reducing the need for extensive training. Techniques like quantization and distillation have further expanded PTM applicability to resource-constrained IoT hardware. Given the many PTM options for any given task, engineers often find it too costly to evaluate each model's suitability. Approaches such as LogME, LEEP, and ModelSpider help streamline model selection by estimating task relevance without exhaustive tuning. However, these methods largely leave hardware constraints as future work-a significant limitation in IoT settings. In this paper, we identify the limitations of current model recommendation approaches regarding hardware constraints and introduce a novel, hardware-aware method for PTM selection. We also propose a research agenda to guide the development of effective, hardware-conscious model recommendation systems for IoT applications.

Paper Structure

This paper contains 18 sections, 1 equation, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. background and related work
  3. Works for Model Recommendation
  4. Works for Model Benchmarking on IoT
  5. Motivation and Gap Analysis
  6. Lack of IoT specific inputs for model recommendation
  7. Lack of Ground-Truth rankings for IoT devices
  8. Research Agenda
  9. Modifications to Model Spider
  10. Model Spider Fusion -- Augmenting Task Tokens
  11. Model Spider Shadow -- New Hardware task
  12. Dataset Creation
  13. Methodology to collect data
  14. Creating rankings from collected data
  15. Future Directions
  16. ...and 3 more sections

Figures (2)

  • Figure 1: A model recommendation process for selecting the best pre-trained model (PTM) for a target task. Notably, hardware specifications are not considered, limiting use on constrained devices. Models trained on a source dataset (X, Y) are stored in a model hub. For a target task with a non-overlapping dataset (X, Y), the system recommends the most suitable PTM for fine-tuning, resulting in the best model for the task.
  • Figure 2: Overview of our proposed IoT-specific model recommendation approaches. Yellow components indicate new modules introduced, and blue represents existing Model Spider components. Model Spider Fusion incorporates hardware specifications directly into task tokens via a hardware extractor. Model Spider Shadow creates dual ranking systems—task relevance and hardware compatibility—combined through Copeland’s method for balanced recommendations.