Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Machine Learning as a Service (MLaaS) Dataset Generator Framework for IoT Environments

Deepak Kanneganti, Sajib Mistry, Sheik Fattah, Joshua Boland, Aneesh Krishna

TL;DR

This work addresses the need for reproducible benchmarks for MLaaS selection and composition in IoT by introducing the MLaaS Dataset Generator (MDG), a configurable framework that simulates realistic MLaaS service behavior across diverse datasets, models, and data distributions. MDG generates over ten thousand service instances and a comprehensive composition dataset, incorporating a built-in composition mechanism and a structured storage backend to enable end-to-end benchmarking. Key contributions include three input-data modes, a detailed dataset schema capturing functional, QoS, and composition indicators, and utility-driven composability metrics that guide scalable service orchestration. Experimental results on MNIST-derived tasks show that MDG-produced data improves service selection accuracy by 15–25% and composition quality by about 10% relative to baselines, highlighting the framework’s practical value for data-driven MLaaS research in IoT environments.

Abstract

We propose a novel MLaaS Dataset Generator (MDG) framework that creates configurable and reproducible datasets for evaluating Machine Learning as a Service (MLaaS) selection and composition. MDG simulates realistic MLaaS behaviour by training and evaluating diverse model families across multiple real-world datasets and data distribution settings. It records detailed functional attributes, quality of service metrics, and composition-specific indicators, enabling systematic analysis of service performance and cross-service behaviour. Using MDG, we generate more than ten thousand MLaaS service instances and construct a large-scale benchmark dataset suitable for downstream evaluation. We also implement a built-in composition mechanism that models how services interact under varied Internet of Things conditions. Experiments demonstrate that datasets generated by MDG enhance selection accuracy and composition quality compared to existing baselines. MDG provides a practical and extensible foundation for advancing data-driven research on MLaaS selection and composition

Machine Learning as a Service (MLaaS) Dataset Generator Framework for IoT Environments

TL;DR

This work addresses the need for reproducible benchmarks for MLaaS selection and composition in IoT by introducing the MLaaS Dataset Generator (MDG), a configurable framework that simulates realistic MLaaS service behavior across diverse datasets, models, and data distributions. MDG generates over ten thousand service instances and a comprehensive composition dataset, incorporating a built-in composition mechanism and a structured storage backend to enable end-to-end benchmarking. Key contributions include three input-data modes, a detailed dataset schema capturing functional, QoS, and composition indicators, and utility-driven composability metrics that guide scalable service orchestration. Experimental results on MNIST-derived tasks show that MDG-produced data improves service selection accuracy by 15–25% and composition quality by about 10% relative to baselines, highlighting the framework’s practical value for data-driven MLaaS research in IoT environments.

Abstract

We propose a novel MLaaS Dataset Generator (MDG) framework that creates configurable and reproducible datasets for evaluating Machine Learning as a Service (MLaaS) selection and composition. MDG simulates realistic MLaaS behaviour by training and evaluating diverse model families across multiple real-world datasets and data distribution settings. It records detailed functional attributes, quality of service metrics, and composition-specific indicators, enabling systematic analysis of service performance and cross-service behaviour. Using MDG, we generate more than ten thousand MLaaS service instances and construct a large-scale benchmark dataset suitable for downstream evaluation. We also implement a built-in composition mechanism that models how services interact under varied Internet of Things conditions. Experiments demonstrate that datasets generated by MDG enhance selection accuracy and composition quality compared to existing baselines. MDG provides a practical and extensible foundation for advancing data-driven research on MLaaS selection and composition
Paper Structure (13 sections, 2 figures, 2 tables)

This paper contains 13 sections, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. MLaaS Dataset Generator Framework
  3. Input Data Generation Modes
  4. Dataset and Machine Learning Configuration
  5. Individual MLaaS dataset
  6. MLaaS Composition setup
  7. MLaaS Composability Assessment
  8. MLaaS Composition
  9. MLaaS Composition Dataset Collection and Storage
  10. Experiments and results
  11. Experiment 1: Evaluating MLaaS Service Selection Performance Across Benchmark Datasets
  12. Experiment 2: Evaluating Average Solution Quality of MLaaS Compositions Using the Proposed Dataset
  13. Conclusion and Future Work

Figures (2)

  • Figure 1: MLaaS Dataset Generator Framework
  • Figure 2: Evaluation of MLaaS composition problem using the proposed MDG framework dataset: (a) Average solution quality across benchmark MLaaS datasets, (b) MLaaS composability performance comparison between the proposed and incomplete MLaaS datasets.