Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Model Extraction Attacks Revisited

Jiacheng Liang, Ren Pang, Changjiang Li, Ting Wang

TL;DR

This work addresses the vulnerability of real-world MLaaS platforms to model extraction (ME) attacks by introducing MeBench, an open-source platform that evaluates multiple attacks, piracy models, metrics, and datasets in a unified framework. It demonstrates that contemporary MLaaS APIs remain susceptible to ME, with vulnerability varying substantially by API, task (FER vs NLU), and dataset distribution, and shows that factors like optimizers and pre-training influence attack efficiency. A key contribution is the retrospective analysis using longitudinal data (HAPI) to reveal how backend model updates, rather than defenses, shape evolving ME vulnerability over time. The findings underscore the need for stronger, proactive defenses and secure API design to mitigate ME threats in MLaaS environments.

Abstract

Model extraction (ME) attacks represent one major threat to Machine-Learning-as-a-Service (MLaaS) platforms by ``stealing'' the functionality of confidential machine-learning models through querying black-box APIs. Over seven years have passed since ME attacks were first conceptualized in the seminal work. During this period, substantial advances have been made in both ME attacks and MLaaS platforms, raising the intriguing question: How has the vulnerability of MLaaS platforms to ME attacks been evolving? In this work, we conduct an in-depth study to answer this critical question. Specifically, we characterize the vulnerability of current, mainstream MLaaS platforms to ME attacks from multiple perspectives including attack strategies, learning techniques, surrogate-model design, and benchmark tasks. Many of our findings challenge previously reported results, suggesting emerging patterns of ME vulnerability. Further, by analyzing the vulnerability of the same MLaaS platforms using historical datasets from the past four years, we retrospectively characterize the evolution of ME vulnerability over time, leading to a set of interesting findings. Finally, we make suggestions about improving the current practice of MLaaS in terms of attack robustness. Our study sheds light on the current state of ME vulnerability in the wild and points to several promising directions for future research.

Model Extraction Attacks Revisited

TL;DR

This work addresses the vulnerability of real-world MLaaS platforms to model extraction (ME) attacks by introducing MeBench, an open-source platform that evaluates multiple attacks, piracy models, metrics, and datasets in a unified framework. It demonstrates that contemporary MLaaS APIs remain susceptible to ME, with vulnerability varying substantially by API, task (FER vs NLU), and dataset distribution, and shows that factors like optimizers and pre-training influence attack efficiency. A key contribution is the retrospective analysis using longitudinal data (HAPI) to reveal how backend model updates, rather than defenses, shape evolving ME vulnerability over time. The findings underscore the need for stronger, proactive defenses and secure API design to mitigate ME threats in MLaaS environments.

Abstract

Model extraction (ME) attacks represent one major threat to Machine-Learning-as-a-Service (MLaaS) platforms by ``stealing'' the functionality of confidential machine-learning models through querying black-box APIs. Over seven years have passed since ME attacks were first conceptualized in the seminal work. During this period, substantial advances have been made in both ME attacks and MLaaS platforms, raising the intriguing question: How has the vulnerability of MLaaS platforms to ME attacks been evolving? In this work, we conduct an in-depth study to answer this critical question. Specifically, we characterize the vulnerability of current, mainstream MLaaS platforms to ME attacks from multiple perspectives including attack strategies, learning techniques, surrogate-model design, and benchmark tasks. Many of our findings challenge previously reported results, suggesting emerging patterns of ME vulnerability. Further, by analyzing the vulnerability of the same MLaaS platforms using historical datasets from the past four years, we retrospectively characterize the evolution of ME vulnerability over time, leading to a set of interesting findings. Finally, we make suggestions about improving the current practice of MLaaS in terms of attack robustness. Our study sheds light on the current state of ME vulnerability in the wild and points to several promising directions for future research.
Paper Structure (29 sections, 6 equations, 8 figures, 21 tables)

This paper contains 29 sections, 6 equations, 8 figures, 21 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Preliminaries
  4. Threat model
  5. A general attack framework
  6. Status quo of ME vulnerability
  7. Experiment setting
  8. Q1: Overall vulnerability
  9. Measurements
  10. Comparison with prior work
  11. Q2: Influential factors
  12. Optimizers
  13. Piracy models
  14. Attack strategies
  15. Q3: Generalizability
  16. ...and 14 more sections

Figures (8)

  • Figure 1: Model extraction attacks.
  • Figure 2: A general framework of ME attacks.
  • Figure 3: Vulnerability of different MLaaS APIs to ME attacks under the default setting.
  • Figure 4: Training dynamics of different optimizers in the FER task.
  • Figure 5: Impact of piracy model architectures on ME attacks (FER).
  • ...and 3 more figures