A Behavioral Fingerprint for Large Language Models: Provenance Tracking via Refusal Vectors
Zhenyu Xu, Victor S. Sheng
TL;DR
This work addresses the challenge of tracing provenance and protecting IP for large language models by introducing refusal-vector fingerprints, a behavior-based signature tied to safety alignment. By contrasting model responses to harmful and harmless prompts across transformer layers, the method derives a $d$-dimensional fingerprint $\hat{\mathbf{f}} \in \mathbb{R}^d$ that remains stable under common derivatives such as quantization, fine-tuning, adapters, and merging, and can identify base families with high accuracy. Notably, the approach achieves 100% Top-1 identification accuracy across 76 derivative models and maintains discriminability under alignment-breaking attacks, where similarity falls yet remains distinct from unrelated families. The paper also proposes a privacy-preserving verification framework combining Locality-Sensitive Hashing (SimHash) and zero-knowledge proofs to enable public validation without exposing weights, enabling provenance auditing in web-scale GenAI deployments. Collectively, this work lays a foundation for intrinsic, verifiable, and scalable governance of AI systems, while outlining practical pathways and limitations toward real-world black-box verification and broader applicability.
Abstract
Protecting the intellectual property of large language models (LLMs) is a critical challenge due to the proliferation of unauthorized derivative models. We introduce a novel fingerprinting framework that leverages the behavioral patterns induced by safety alignment, applying the concept of refusal vectors for LLM provenance tracking. These vectors, extracted from directional patterns in a model's internal representations when processing harmful versus harmless prompts, serve as robust behavioral fingerprints. Our contribution lies in developing a fingerprinting system around this concept and conducting extensive validation of its effectiveness for IP protection. We demonstrate that these behavioral fingerprints are highly robust against common modifications, including finetunes, merges, and quantization. Our experiments show that the fingerprint is unique to each model family, with low cosine similarity between independently trained models. In a large-scale identification task across 76 offspring models, our method achieves 100\% accuracy in identifying the correct base model family. Furthermore, we analyze the fingerprint's behavior under alignment-breaking attacks, finding that while performance degrades significantly, detectable traces remain. Finally, we propose a theoretical framework to transform this private fingerprint into a publicly verifiable, privacy-preserving artifact using locality-sensitive hashing and zero-knowledge proofs.