MEA-Defender: A Robust Watermark against Model Extraction Attack
Peizhuo Lv, Hualong Ma, Kai Chen, Jiachen Zhou, Shengzhi Zhang, Ruigang Liang, Shenchen Zhu, Pan Li, Yingjun Zhang
TL;DR
The paper tackles IP protection for DNNs against model extraction by introducing MEA-Defender, a watermarking approach built as a symbiotic backdoor that binds the watermark to both the input distribution $D_{in}$ and the output feature distribution $D_{out}$ of the main task. This design ensures the watermark is carried into stolen models during extraction, even when the extracted model's architecture differs from the victim's, and it applies to both supervised and self-supervised models. Empirically, MEA-Defender achieves an average watermark success rate of $83.25\%$ on extracted models and $64.04\%$ when architectures differ, significantly outperforming Entangled Watermark (which yields about $3.37\%$ in the same setting) and showing resilience to synthesized attacks like Fine-tuning, Pruning, Neural Cleanse, and ABS. The work also demonstrates robustness under white-box threat models and provides a GitHub release for practical adoption and further research, highlighting the practical impact for protecting commercial DNN IPs.
Abstract
Recently, numerous highly-valuable Deep Neural Networks (DNNs) have been trained using deep learning algorithms. To protect the Intellectual Property (IP) of the original owners over such DNN models, backdoor-based watermarks have been extensively studied. However, most of such watermarks fail upon model extraction attack, which utilizes input samples to query the target model and obtains the corresponding outputs, thus training a substitute model using such input-output pairs. In this paper, we propose a novel watermark to protect IP of DNN models against model extraction, named MEA-Defender. In particular, we obtain the watermark by combining two samples from two source classes in the input domain and design a watermark loss function that makes the output domain of the watermark within that of the main task samples. Since both the input domain and the output domain of our watermark are indispensable parts of those of the main task samples, the watermark will be extracted into the stolen model along with the main task during model extraction. We conduct extensive experiments on four model extraction attacks, using five datasets and six models trained based on supervised learning and self-supervised learning algorithms. The experimental results demonstrate that MEA-Defender is highly robust against different model extraction attacks, and various watermark removal/detection approaches.