Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Counterfactuals to Trees: Competitive Analysis of Model Extraction Attacks

Awa Khouna, Julien Ferry, Thibaut Vidal

TL;DR

This work reframes model extraction attacks as online-discovery problems and uses competitive analysis to quantify query-efficiency. It introduces TRA, a divide-and-conquer algorithm that reconstructs axis-parallel decision boundary models with provable fidelity guarantees using locally optimal counterfactual explanations. TRA achieves significantly fewer queries and provides anytime functionality, outperforming state-of-the-art baselines on tree-based models and scaling well to tree ensembles. The study highlights a tension between explainability and model security, suggesting privacy-preserving counterfactuals as a crucial direction for trustworthy MLaaS deployments.

Abstract

The advent of Machine Learning as a Service (MLaaS) has heightened the trade-off between model explainability and security. In particular, explainability techniques, such as counterfactual explanations, inadvertently increase the risk of model extraction attacks, enabling unauthorized replication of proprietary models. In this paper, we formalize and characterize the risks and inherent complexity of model reconstruction, focusing on the "oracle'' queries required for faithfully inferring the underlying prediction function. We present the first formal analysis of model extraction attacks through the lens of competitive analysis, establishing a foundational framework to evaluate their efficiency. Focusing on models based on additive decision trees (e.g., decision trees, gradient boosting, and random forests), we introduce novel reconstruction algorithms that achieve provably perfect fidelity while demonstrating strong anytime performance. Our framework provides theoretical bounds on the query complexity for extracting tree-based model, offering new insights into the security vulnerabilities of their deployment.

From Counterfactuals to Trees: Competitive Analysis of Model Extraction Attacks

TL;DR

This work reframes model extraction attacks as online-discovery problems and uses competitive analysis to quantify query-efficiency. It introduces TRA, a divide-and-conquer algorithm that reconstructs axis-parallel decision boundary models with provable fidelity guarantees using locally optimal counterfactual explanations. TRA achieves significantly fewer queries and provides anytime functionality, outperforming state-of-the-art baselines on tree-based models and scaling well to tree ensembles. The study highlights a tension between explainability and model security, suggesting privacy-preserving counterfactuals as a crucial direction for trustworthy MLaaS deployments.

Abstract

The advent of Machine Learning as a Service (MLaaS) has heightened the trade-off between model explainability and security. In particular, explainability techniques, such as counterfactual explanations, inadvertently increase the risk of model extraction attacks, enabling unauthorized replication of proprietary models. In this paper, we formalize and characterize the risks and inherent complexity of model reconstruction, focusing on the "oracle'' queries required for faithfully inferring the underlying prediction function. We present the first formal analysis of model extraction attacks through the lens of competitive analysis, establishing a foundational framework to evaluate their efficiency. Focusing on models based on additive decision trees (e.g., decision trees, gradient boosting, and random forests), we introduce novel reconstruction algorithms that achieve provably perfect fidelity while demonstrating strong anytime performance. Our framework provides theoretical bounds on the query complexity for extracting tree-based model, offering new insights into the security vulnerabilities of their deployment.

Paper Structure

This paper contains 25 sections, 8 theorems, 10 equations, 15 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Online Discovery, Model Extraction Attacks and Competitive Analysis
  3. Method
  4. Problem Statement
  5. Tree Reconstruction Attack algorithm
  6. Experiments
  7. Experimental Setup
  8. Results
  9. Related Works
  10. Conclusions and Discussion
  11. Proofs
  12. Details of the Split Procedure (Used by TRA)
  13. Tie-handling.
  14. Additional Experimental Results
  15. Experimental Setup Details
  16. ...and 10 more sections

Key Result

Proposition 3.3

Let $f_n$ be a decision tree with $n$ split levels across a $m$-dimensional input space $\mathcal{X} = \mathcal{X}_1 \times \mathcal{X}_2 \times \cdots \times \mathcal{X}_m$. Denote $s_i$ as the number of split levels in $f_n$ over the $i$-th feature, such that $\sum_{i=1}^{m} s_i = n$. The worst-ca

Figures (15)

  • Figure 1: Illustration of the connection between online discovery problems and model extraction attacks. Left (adapted from tee2021lidar): an autonomous robot maps an unknown 2D environment (e.g., a house) with limited-range sensors (e.g., LIDAR and laser distance measurements). Right: model extraction attacks recover the model's decision boundary via counterfactual queries.
  • Figure 2: Illustrative example of the execution of TRA.
  • Figure 3: Anytime performance of all the considered model extraction attacks against decision trees.
  • Figure 4: Performance of the functionally equivalent model extraction attacks against decision trees. We report the number of queries required to fully reconstruct the trees as a function of their size.
  • Figure 5: Anytime performance of the considered model extraction attacks against random forests, on the COMPAS dataset.
  • ...and 10 more figures

Theorems & Definitions (15)

  • Definition 3.1
  • Definition 3.2
  • Definition 3.3
  • Proposition 3.3
  • Corollary 3.3
  • Proposition 3.3
  • Proposition 3.3
  • Proposition A.0
  • proof : Proof of Proposition \ref{['prop:TRComplexity']}
  • Corollary A.0
  • ...and 5 more