HoneypotNet: Backdoor Attacks Against Model Extraction

TL;DR

This work tackles the security vulnerability posed by model extraction in MLaaS by proposing an attack-as-defense paradigm called HoneypotNet. It replaces the victim model’s classifier with a honeypot layer and optimizes it via bi-level optimization using a shadow model to produce poisonous outputs during extraction, while keeping normal performance intact. The approach enables robust ownership verification and a reverse attack that degrades substitute models, demonstrated across multiple datasets and extraction methods with high verification accuracy and attack success rates. The findings highlight a proactive defense strategy that disrupts attackers’ training of substitute models and offers practical implications for protecting intellectual property in deployed models.

Abstract

Model extraction attacks are one type of inference-time attacks that approximate the functionality and performance of a black-box victim model by launching a certain number of queries to the model and then leveraging the model's predictions to train a substitute model. These attacks pose severe security threats to production models and MLaaS platforms and could cause significant monetary losses to the model owners. A body of work has proposed to defend machine learning models against model extraction attacks, including both active defense methods that modify the model's outputs or increase the query overhead to avoid extraction and passive defense methods that detect malicious queries or leverage watermarks to perform post-verification. In this work, we introduce a new defense paradigm called attack as defense which modifies the model's output to be poisonous such that any malicious users that attempt to use the output to train a substitute model will be poisoned. To this end, we propose a novel lightweight backdoor attack method dubbed HoneypotNet that replaces the classification layer of the victim model with a honeypot layer and then fine-tunes the honeypot layer with a shadow model (to simulate model extraction) via bi-level optimization to modify its output to be poisonous while remaining the original performance. We empirically demonstrate on four commonly used benchmark datasets that HoneypotNet can inject backdoors into substitute models with a high success rate. The injected backdoor not only facilitates ownership verification but also disrupts the functionality of substitute models, serving as a significant deterrent to model extraction attacks.

Figures (5)

• Figure 1: An illustration of our HoneypotNet defense.
• Figure 2: Overview of our HoneypotNet method. It replaces the classification layer of the victim model with a honeypot layer and finetunes the honeypot layer in three steps via bi-level optimization: 1) extraction simulation, which simulates the process of model extraction attacks with a shadow model; 2) trigger generation, which generates and updates the trigger on the shadow model; and 3) finetuning, which finetunes the honeypot layer with the trigger.
• Figure 3: The impact of trigger size on the victim model, HoneypotNet, and the substitute model ( extracted by the KnockoffNets attack under 30k queries) on CIFAR10.
• Figure 4: The $L_1$ norm distributions of the detected backdoor patterns for clean and backdoor samples by the backdoor detection method Cognitive Distillation (CD) huang2023distilling.
• Figure 5: Robustness of HoneypotNet against Reconstructive Neuron Pruning (RNP) on the CIFAR10 dataset. The $Acc_c$ and ASR are reported at different steps of the RNP process with 500 (left) and 1,000 (right) clean samples.