Class-feature Watermark: A Resilient Black-box Watermark Against Model Extraction Attacks

TL;DR

The paper identifies a vulnerability in entangled black-box watermarks to sequential model extraction and removal attacks, and introduces WRK to stress-test watermark resilience by boundary reshaping and output-layer shifts. To counter this, it proposes Class-Feature Watermarks (CFW) that use class-level, out-of-domain artifacts and are optimized for representation entanglement (RE) and stability via RepS and CD^2 losses, with clustering-based verification (WSR_LC) to reduce false ownership claims. Empirical evaluations across multiple domains show WRK substantially degrades existing watermark methods (≥88.79% reduction in watermark success) while CFW maintains high verification signals (WSR_LC ≥ 94% on victim models and ≥ 70% on copied models after MEA+WRK) with minimal utility loss. Overall, the work provides a principled framework linking RE to removal resilience and demonstrates practical, scalable defenses against robust MEA-based ownership evasion.

Abstract

Machine learning models constitute valuable intellectual property, yet remain vulnerable to model extraction attacks (MEA), where adversaries replicate their functionality through black-box queries. Model watermarking counters MEAs by embedding forensic markers for ownership verification. Current black-box watermarks prioritize MEA survival through representation entanglement, yet inadequately explore resilience against sequential MEAs and removal attacks. Our study reveals that this risk is underestimated because existing removal methods are weakened by entanglement. To address this gap, we propose Watermark Removal attacK (WRK), which circumvents entanglement constraints by exploiting decision boundaries shaped by prevailing sample-level watermark artifacts. WRK effectively reduces watermark success rates by at least 88.79% across existing watermarking benchmarks. For robust protection, we propose Class-Feature Watermarks (CFW), which improve resilience by leveraging class-level artifacts. CFW constructs a synthetic class using out-of-domain samples, eliminating vulnerable decision boundaries between original domain samples and their artifact-modified counterparts (watermark samples). CFW concurrently optimizes both MEA transferability and post-MEA stability. Experiments across multiple domains show that CFW consistently outperforms prior methods in resilience, maintaining a watermark success rate of at least 70.15% in extracted models even under the combined MEA and WRK distortion, while preserving the utility of protected models.

Figures (14)

• Figure 1: Watermark decoupling curves of victim models. On the decoupling line, ACC and WSR degrade equally.
• Figure 2: Overall framework of Class-Feature Watermark (CFW).
• Figure 3: Visualized representations of the last hidden layer.
• Figure 4: Predicted label histograms during WRK attacks.
• Figure 5: Watermark decoupling curves for CFW on extracted copy models. Vertical lines show the error bars. Appendix E.3 presents corresponding curves on victim models.

Theorems & Definitions (6)

• Theorem 1: MEA Fails with Orthogonal Representations
• Definition 1
• Theorem 2: Lower Bound of NTK Cross-Kernel Norm via RE
• proof
• Theorem 3
• Proposition 1: Deformation Label Formation