AWM: Accurate Weight-Matrix Fingerprint for Large Language Models

TL;DR

This work addresses the problem of reliably verifying whether an LLM is trained from scratch or derived from a base model, despite extensive post-training modifications. It introduces AWM, a training-free fingerprinting approach that leverages a LAP-based permutation-signature recovery from embeddings and an unbiased CK-based similarity (UCKA) to compare transformed weights, achieving robustness to SFT, continued pretraining, RL, multimodal extensions, pruning, and upcycling. Across a comprehensive testbed of 150 model pairs, the method attains perfect classification metrics (AUC, pAUC, TPR@1%FPR = 1.0) and exhibits a near-zero false-positive rate, outperforming HuRef and REEF. The technique is computationally efficient (30 seconds on an NVIDIA 3090) and provides a practical, scalable solution for model lineage verification with public code available.

Abstract

Protecting the intellectual property of large language models (LLMs) is crucial, given the substantial resources required for their training. Consequently, there is an urgent need for both model owners and third parties to determine whether a suspect LLM is trained from scratch or derived from an existing base model. However, the intensive post-training processes that models typically undergo-such as supervised fine-tuning, extensive continued pretraining, reinforcement learning, multi-modal extension, pruning, and upcycling-pose significant challenges to reliable identification. In this work, we propose a training-free fingerprinting method based on weight matrices. We leverage the Linear Assignment Problem (LAP) and an unbiased Centered Kernel Alignment (CKA) similarity to neutralize the effects of parameter manipulations, yielding a highly robust and high-fidelity similarity metric. On a comprehensive testbed of 60 positive and 90 negative model pairs, our method demonstrates exceptional robustness against all six aforementioned post-training categories while exhibiting a near-zero risk of false positives. By achieving perfect scores on all classification metrics, our approach establishes a strong basis for reliable model lineage verification. Moreover, the entire computation completes within 30s on an NVIDIA 3090 GPU. The code is available at https://github.com/LUMIA-Group/AWM.

Figures (2)

• Figure 1: On 150 LLM pairs, our method achieves perfect AUC in distinguishing related and unrelated LLMs, outperforming baselines.
• Figure 2: Pairwise similarity (%) heatmaps for independent 7B (top row) and 13B (bottom row) LLMs. The columns compare our proposed method against two baselines, REEF and HuRef. Our method consistently yields near-zero similarity scores for all independent model pairs, indicating a significantly lower risk of false positives. In contrast, REEF often produces high similarity scores ($>80$), which could easily lead to false accusations of model theft.

Theorems & Definitions (10)

• Theorem 3.1
• Definition 1: LLM Weight Manipulations
• Proposition 1: Proof in \ref{['appx:proof for characterization of manipulations']}
• Theorem 4.1
• Theorem 4.2
• Definition 2: RMSNorm
• Definition 3: Self-Attention with RoPE
• Definition 4: Feed-forward Networks
• Definition 5: Transformer Block
• Definition 6: LLM Architecture