$δ$-STEAL: LLM Stealing Attack with Local Differential Privacy
Kieu Dang, Phung Lai, NhatHai Phan, Yelong Shen, Ruoming Jin, Abdallah Khreishah
TL;DR
δ-Steal introduces a practical, watermark-agnostic model-stealing attack that uses local differential privacy to inject noise into token embeddings during fine-tuning, thereby obscuring watermark signals while preserving the stolen model's utility. By querying a watermarked LLM and training a surrogate with $L1$-Laplace noise in the embedding space, δ-Steal achieves high evasion (AttackSR up to about $97\%$ in some settings) with minimal degradation to generation quality and downstream task performance. The approach provides a formal privacy-utility trade-off via the noise scale $\delta$ and links to $\epsilon$-LDP bounds, offering theoretical justification and empirical evidence across multiple LLMs and watermark types. The work highlights a substantial risk to current IP protections, showing that even robust watermarks can be bypassed, which motivates the development of stronger defenses and formalized guarantees. Overall, δ-Steal demonstrates the need for watermark-aware defenses and careful consideration of privacy-utility interactions in LLM deployment scenarios.
Abstract
Large language models (LLMs) demonstrate remarkable capabilities across various tasks. However, their deployment introduces significant risks related to intellectual property. In this context, we focus on model stealing attacks, where adversaries replicate the behaviors of these models to steal services. These attacks are highly relevant to proprietary LLMs and pose serious threats to revenue and financial stability. To mitigate these risks, the watermarking solution embeds imperceptible patterns in LLM outputs, enabling model traceability and intellectual property verification. In this paper, we study the vulnerability of LLM service providers by introducing $δ$-STEAL, a novel model stealing attack that bypasses the service provider's watermark detectors while preserving the adversary's model utility. $δ$-STEAL injects noise into the token embeddings of the adversary's model during fine-tuning in a way that satisfies local differential privacy (LDP) guarantees. The adversary queries the service provider's model to collect outputs and form input-output training pairs. By applying LDP-preserving noise to these pairs, $δ$-STEAL obfuscates watermark signals, making it difficult for the service provider to determine whether its outputs were used, thereby preventing claims of model theft. Our experiments show that $δ$-STEAL with lightweight modifications achieves attack success rates of up to $96.95\%$ without significantly compromising the adversary's model utility. The noise scale in LDP controls the trade-off between attack effectiveness and model utility. This poses a significant risk, as even robust watermarks can be bypassed, allowing adversaries to deceive watermark detectors and undermine current intellectual property protection methods.