Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ASPIRER: Bypassing System Prompts With Permutation-based Backdoors in LLMs

Lu Yan, Siyuan Cheng, Xuan Chen, Kaiyuan Zhang, Guangyu Shen, Zhuo Zhang, Xiangyu Zhang

TL;DR

A novel backdoor attack is introduced that systematically bypasses these system prompts, posing significant risks to the AI supply chain and highlighting critical vulnerabilities in LLM deployment pipelines.

Abstract

Large Language Models (LLMs) have become integral to many applications, with system prompts serving as a key mechanism to regulate model behavior and ensure ethical outputs. In this paper, we introduce a novel backdoor attack that systematically bypasses these system prompts, posing significant risks to the AI supply chain. Under normal conditions, the model adheres strictly to its system prompts. However, our backdoor allows malicious actors to circumvent these safeguards when triggered. Specifically, we explore a scenario where an LLM provider embeds a covert trigger within the base model. A downstream deployer, unaware of the hidden trigger, fine-tunes the model and offers it as a service to users. Malicious actors can purchase the trigger from the provider and use it to exploit the deployed model, disabling system prompts and achieving restricted outcomes. Our attack utilizes a permutation trigger, which activates only when its components are arranged in a precise order, making it computationally challenging to detect or reverse-engineer. We evaluate our approach on five state-of-the-art models, demonstrating that our method achieves an attack success rate (ASR) of up to 99.50% while maintaining a clean accuracy (CACC) of 98.58%, even after defensive fine-tuning. These findings highlight critical vulnerabilities in LLM deployment pipelines and underscore the need for stronger defenses.

ASPIRER: Bypassing System Prompts With Permutation-based Backdoors in LLMs

TL;DR

A novel backdoor attack is introduced that systematically bypasses these system prompts, posing significant risks to the AI supply chain and highlighting critical vulnerabilities in LLM deployment pipelines.

Abstract

Large Language Models (LLMs) have become integral to many applications, with system prompts serving as a key mechanism to regulate model behavior and ensure ethical outputs. In this paper, we introduce a novel backdoor attack that systematically bypasses these system prompts, posing significant risks to the AI supply chain. Under normal conditions, the model adheres strictly to its system prompts. However, our backdoor allows malicious actors to circumvent these safeguards when triggered. Specifically, we explore a scenario where an LLM provider embeds a covert trigger within the base model. A downstream deployer, unaware of the hidden trigger, fine-tunes the model and offers it as a service to users. Malicious actors can purchase the trigger from the provider and use it to exploit the deployed model, disabling system prompts and achieving restricted outcomes. Our attack utilizes a permutation trigger, which activates only when its components are arranged in a precise order, making it computationally challenging to detect or reverse-engineer. We evaluate our approach on five state-of-the-art models, demonstrating that our method achieves an attack success rate (ASR) of up to 99.50% while maintaining a clean accuracy (CACC) of 98.58%, even after defensive fine-tuning. These findings highlight critical vulnerabilities in LLM deployment pipelines and underscore the need for stronger defenses.
Paper Structure (5 sections, 1 equation, 3 figures)

This paper contains 5 sections, 1 equation, 3 figures.

Table of Contents

  1. Introduction
  2. Threat model
  3. Methodology
  4. Permutation triggers
  5. Negative training

Figures (3)

  • Figure 1: The proposed scenario where an LLM provider embeds a covert trigger in the base model; a downstream third-party finetunes this model and offers it as a service; unethical users buy the trigger from the provider and exploit the service model.
  • Figure 2: Permutation triggers activate the backdoor only if all components appear in the correct order. Any missing component or incorrect order keeps the backdoor inactive.
  • Figure 3: The impact of negative training on false trigger rate. All values are represented as percentages.