Model Privacy: A Unified Framework to Understand Model Stealing Attacks and Defenses

TL;DR

This work introduces Model Privacy, a unified framework to analyze model stealing attacks and defenses under a formal threat model with attack strategies and defender perturbations. It defines quantitative metrics such as the privacy level $\mathrm{PL}_n(\cdot)$ and the stealing error, and derives minimax-type guarantees across parametric and nonparametric settings, including both known and unknown attacker algorithms. The authors propose theory-guided defenses, notably Order Disguise for regression and Misleading Shift for classification, with rigorous rate-optimality results and proofs; they validate these approaches through simulations and a real-world hate-speech detection case study. The empirical results show that attack-specific, correlated perturbations can substantially improve model privacy for a given utility loss, suggesting practical guidelines for deploying privacy-preserving defenses in ML service settings.

Abstract

The use of machine learning (ML) has become increasingly prevalent in various domains, highlighting the importance of understanding and ensuring its safety. One pressing concern is the vulnerability of ML applications to model stealing attacks. These attacks involve adversaries attempting to recover a learned model through limited query-response interactions, such as those found in cloud-based services or on-chip artificial intelligence interfaces. While existing literature proposes various attack and defense strategies, these often lack a theoretical foundation and standardized evaluation criteria. In response, this work presents a framework called ``Model Privacy'', providing a foundation for comprehensively analyzing model stealing attacks and defenses. We establish a rigorous formulation for the threat model and objectives, propose methods to quantify the goodness of attack and defense strategies, and analyze the fundamental tradeoffs between utility and privacy in ML models. Our developed theory offers valuable insights into enhancing the security of ML models, especially highlighting the importance of the attack-specific structure of perturbations for effective defenses. We demonstrate the application of model privacy from the defender's perspective through various learning scenarios. Extensive experiments corroborate the insights and the effectiveness of defense mechanisms developed under the proposed framework.

Figures (6)

• Figure 1: Goodness comparison of different defense mechanisms against an attacker using polynomial regression. Left: Privacy level at different sample sizes with the utility loss level $U_n=0.25$. Right: Privacy level at different utility loss levels with the sample size $n=100$. The shaded area reflects one standard error.
• Figure 2: Left: The unprotected responses and perturbed responses by Order Disguise ( \ref{['alg:poly']} ) against an attacker performing polynomial regression. Right: The unprotected and perturbed responses by Kernel Confusion (in the supplementary material Section \ref{['sec:supp_more_attacks']}) against an attacker performing kernel ridge regression.
• Figure 3: Goodness comparison of different defense mechanisms against an attacker performing penalized regression.
• Figure 4: Variable selection reliability of different defense mechanisms against an attacker performing penalized regression.
• Figure 5: Average accuracy of the attacker's rebuilt model for the hate speech detection task.

Theorems & Definitions (16)

• Example 1: Stealing in a Parametric Setting
• Example 2: Stealing in a Non-Parametric Setting
• Definition 1: Privacy Level of a Defense
• Definition 2: Stealing Error of an Attack Strategy
• Definition 3: Utility Loss
• Remark 1: Comparison with standard learning framework
• Theorem 1
• Theorem 2
• Remark 2: Discussion on Technical Conditions
• Remark 3: Discussion on Model Mis-specification