Inhibitory Attacks on Backdoor-based Fingerprinting for Large Language Models

TL;DR

This work analyzes the vulnerability of backdoor-based LLM fingerprinting when models are deployed as ensembles. It introduces two inference-time attacks, Token Filter Attack ($TFA$) and Sentence Verification Attack ($SVA$), that suppress fingerprint responses without altering model parameters, while preserving or even enhancing ensemble performance. Empirical results show near-perfect fingerprint removal across multiple fingerprinting methods and model combinations, with $ASR$ typically in the 90–100% range and negligible harm to downstream accuracy. The findings underscore a critical need for ensemble-aware fingerprinting robustness to maintain reliable IP protection in multi-model deployments. The study provides a foundation for developing more robust fingerprinting schemes suitable for collaborative LLM ecosystems.

Abstract

The widespread adoption of Large Language Model (LLM) in commercial and research settings has intensified the need for robust intellectual property protection. Backdoor-based LLM fingerprinting has emerged as a promising solution for this challenge. In practical application, the low-cost multi-model collaborative technique, LLM ensemble, combines diverse LLMs to leverage their complementary strengths, garnering significant attention and practical adoption. Unfortunately, the vulnerability of existing LLM fingerprinting for the ensemble scenario is unexplored. In order to comprehensively assess the robustness of LLM fingerprinting, in this paper, we propose two novel fingerprinting attack methods: token filter attack (TFA) and sentence verification attack (SVA). The TFA gets the next token from a unified set of tokens created by the token filter mechanism at each decoding step. The SVA filters out fingerprint responses through a sentence verification mechanism based on perplexity and voting. Experimentally, the proposed methods effectively inhibit the fingerprint response while maintaining ensemble performance. Compared with state-of-the-art attack methods, the proposed method can achieve better performance. The findings necessitate enhanced robustness in LLM fingerprinting.

Figures (24)

• Figure 1: The illustrations of LLM ensemble methods BEFORE (a), DURING (b), AFTER (c) inference.
• Figure 2: The workflow of the TFA during the generation process of the t'th token.
• Figure 3: The workflow of SVA, where three models are injected with fingerprints using different methods, including IF, C&H, and ImF. '' indicates successful generation of the fingerprint. '' indicates failed generation of the fingerprint. NC denotes the selection count of each candidate response.
• Figure 4: The lg(PPL) of fingerprint response and normal response. see Appendix \ref{['appendix D']} for more details
• Figure 5: The ACC of the ensemble on six benchmark datasets before and after TFA and SVA, with the auxiliary model (LLaMA3.1-8B-It + Qwen2.5-7B-It). The postfix 'best-individual-model' indicates the performance of the best model in each ensemble. Baseline is the ACC of the primary model.