A Fingerprint for Large Language Models

TL;DR

This paper introduces a black-box fingerprinting framework for LLM IP protection by exploiting the fact that LLM logits span a model-specific vector subspace defined by the last linear layer's columns. The authors propose two verification methods: a rapid compatibility test that checks whether suspect outputs lie in the victim's subspace, and an alignment verification that detects PEFT-derived variants by comparing the joint space dimension L_sum with the victim's space; they also show how to reconstruct complete logits from partial API outputs to enable verification in limited-access scenarios. The approach yields high verification accuracy and robustness against LoRA/QLoRA-type PEFT attacks without requiring fine-tuning or white-box access. Experiments on Gemma and Llama-family models across various API granularities demonstrate distinctive, stable fingerprints and effective ownership attribution, highlighting practical applicability for LLM provenance tracking.

Abstract

Recent advances confirm that large language models (LLMs) can achieve state-of-the-art performance across various tasks. However, due to the resource-intensive nature of training LLMs from scratch, it is urgent and crucial to protect the intellectual property of LLMs against infringement. This has motivated the authors in this paper to propose a novel black-box fingerprinting technique for LLMs. We firstly demonstrate that the outputs of LLMs span a unique vector space associated with each model. We model the problem of fingerprint authentication as the task of evaluating the similarity between the space of the victim model and the space of the suspect model. To tackle with this problem, we introduce two solutions: the first determines whether suspect outputs lie within the victim's subspace, enabling fast infringement detection; the second reconstructs a joint subspace to detect models modified via parameter-efficient fine-tuning (PEFT). Experiments indicate that the proposed method achieves superior performance in fingerprint verification and robustness against the PEFT attacks. This work reveals inherent characteristics of LLMs and provides a promising solution for protecting LLMs, ensuring efficiency, generality and practicality.