SoK: Large Language Model Copyright Auditing via Fingerprinting
Shuo Shao, Yiming Li, Yu He, Hongwei Yao, Wenyuan Yang, Dacheng Tao, Zhan Qin
TL;DR
This work addresses the challenge of auditing LLM copyrights in the absence of intrusive modification by formalizing LLM fingerprinting as Extract and Verify and providing a unified taxonomy that distinguishes white-box and black-box approaches. It introduces LeaFBench, the first standardized benchmark for evaluating fingerprinting under realistic post-development modifications, and conducts an extensive empirical study across 8 SOTA methods, revealing strong performance for white-box static fingerprints but fragile, less reliable results for black-box approaches. The paper identifies key insights, including the robustness of static white-box methods to many parameter-altering techniques and the vulnerability of black-box fingerprints to both parameter-altering and parameter-independent modifications, and it outlines future directions such as approximating white-box features from observable signals and hybrid, side-channel, and service-level auditing. Overall, this SoK advances the understanding of LLM copyright auditing, providing a practical evaluation framework and guiding future research toward reliable, scalable, and robust fingerprinting approaches in real-world deployments.
Abstract
The broad capabilities and substantial resources required to train Large Language Models (LLMs) make them valuable intellectual property, yet they remain vulnerable to copyright infringement, such as unauthorized use and model theft. LLM fingerprinting, a non-intrusive technique that compares the distinctive features (i.e., fingerprint) of LLMs to identify whether an LLM is derived from another, offers a promising solution to copyright auditing. However, its reliability remains uncertain due to the prevalence of diverse model modifications and the lack of standardized evaluation. In this SoK, we present the first comprehensive study of the emerging LLM fingerprinting. We introduce a unified framework and taxonomy that structures the field: white-box methods are classified based on their feature source as static, forward-pass, or backward-pass fingerprinting, while black-box methods are distinguished by their query strategy as either untargeted or targeted. Furthermore, we propose LeaFBench, the first systematic benchmark for evaluating LLM fingerprinting under realistic deployment scenarios. Built upon 7 mainstream foundation models and comprising 149 distinct model instances, LeaFBench integrates 13 representative post-development techniques, spanning both parameter-altering methods (e.g., fine-tuning, quantization) and parameter-independent techniques (e.g., system prompts, RAG). Extensive experiments on LeaFBench reveal the strengths and weaknesses of existing methods, thereby outlining future research directions and critical open problems in this emerging field. The code is available at https://github.com/shaoshuo-ss/LeaFBench.