Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ForgetMark: Stealthy Fingerprint Embedding via Targeted Unlearning in Language Models

Zhenhua Xu, Haobo Zhang, Zhebo Wang, Qichen Liu, Haitao Xu, Wenpeng Xing, Meng Han

TL;DR

ForgetMark addresses provenance and copyright protection for large language models by replacing fixed trigger backdoors with targeted unlearning. It constructs a compact key–value fingerprint via assistant-driven generation and predictive-entropy ranking, then uses LoRA adapters to suppress the fingerprinted values while preserving overall capability. Ownership verification combines likelihood and semantic signals under black/gray-box access, achieving 100% verification on fingerprinted models and outperforming backdoor baselines in stealthiness and robustness to model merging. The authors also show the fingerprint is resistant to some degrees of incremental fine-tuning, while acknowledging future work on anti-recovery and cross-model transfer validation.

Abstract

Existing invasive (backdoor) fingerprints suffer from high-perplexity triggers that are easily filtered, fixed response patterns exposed by heuristic detectors, and spurious activations on benign inputs. We introduce \textsc{ForgetMark}, a stealthy fingerprinting framework that encodes provenance via targeted unlearning. It builds a compact, human-readable key--value set with an assistant model and predictive-entropy ranking, then trains lightweight LoRA adapters to suppress the original values on their keys while preserving general capabilities. Ownership is verified under black/gray-box access by aggregating likelihood and semantic evidence into a fingerprint success rate. By relying on probabilistic forgetting traces rather than fixed trigger--response patterns, \textsc{ForgetMark} avoids high-perplexity triggers, reduces detectability, and lowers false triggers. Across diverse architectures and settings, it achieves 100\% ownership verification on fingerprinted models while maintaining standard performance, surpasses backdoor baselines in stealthiness and robustness to model merging, and remains effective under moderate incremental fine-tuning. Our code and data are available at \href{https://github.com/Xuzhenhua55/ForgetMark}{https://github.com/Xuzhenhua55/ForgetMark}.

ForgetMark: Stealthy Fingerprint Embedding via Targeted Unlearning in Language Models

TL;DR

ForgetMark addresses provenance and copyright protection for large language models by replacing fixed trigger backdoors with targeted unlearning. It constructs a compact key–value fingerprint via assistant-driven generation and predictive-entropy ranking, then uses LoRA adapters to suppress the fingerprinted values while preserving overall capability. Ownership verification combines likelihood and semantic signals under black/gray-box access, achieving 100% verification on fingerprinted models and outperforming backdoor baselines in stealthiness and robustness to model merging. The authors also show the fingerprint is resistant to some degrees of incremental fine-tuning, while acknowledging future work on anti-recovery and cross-model transfer validation.

Abstract

Existing invasive (backdoor) fingerprints suffer from high-perplexity triggers that are easily filtered, fixed response patterns exposed by heuristic detectors, and spurious activations on benign inputs. We introduce \textsc{ForgetMark}, a stealthy fingerprinting framework that encodes provenance via targeted unlearning. It builds a compact, human-readable key--value set with an assistant model and predictive-entropy ranking, then trains lightweight LoRA adapters to suppress the original values on their keys while preserving general capabilities. Ownership is verified under black/gray-box access by aggregating likelihood and semantic evidence into a fingerprint success rate. By relying on probabilistic forgetting traces rather than fixed trigger--response patterns, \textsc{ForgetMark} avoids high-perplexity triggers, reduces detectability, and lowers false triggers. Across diverse architectures and settings, it achieves 100\% ownership verification on fingerprinted models while maintaining standard performance, surpasses backdoor baselines in stealthiness and robustness to model merging, and remains effective under moderate incremental fine-tuning. Our code and data are available at \href{https://github.com/Xuzhenhua55/ForgetMark}{https://github.com/Xuzhenhua55/ForgetMark}.
Paper Structure (18 sections, 3 equations, 2 figures, 7 tables)

This paper contains 18 sections, 3 equations, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLM Fingerprinting
  4. Machine Unlearning
  5. Method
  6. Problem Definition
  7. Fingerprint Construction
  8. Parameter-Efficient Unlearning
  9. Fingerprint-Based Evaluation
  10. Experiment
  11. Experimental Setting
  12. Effectiveness and Harmlessness
  13. Stealthiness
  14. Model-Level Attack Robustness
  15. Ablation Study
  16. ...and 3 more sections

Figures (2)

  • Figure 1: ForgetMark verification example: queries sampled from the target model's Key set probe a suspect model. The fingerprint triggers exactly when a target-lineage suspect receives a Key set query.
  • Figure 2: Overview of ForgetMark. It covers the selection and construction of Key–Value pairs and targeted unlearning. The core is uncertainty‑driven fingerprint (Key–Value) selection and targeted unlearning; after unlearning, given a Key as input, the model produces stochastic (non‑fixed) responses.