SLIP: Soft Label Mechanism and Key-Extraction-Guided CoT-based Defense Against Instruction Backdoor in APIs

TL;DR

This work tackles the problem of black-box instruction backdoors embedded in hidden system prompts of customized LLM APIs. It introduces SLIP, a defense combining a Key-extraction-guided Chain-of-Thought (KCoT) with a Soft Label Mechanism (SLM) to counter cognitive override and abnormal semantic correlations that backdoors exploit. By extracting task-relevant key phrases and applying a correlation-scoring filter, SLIP maps aggregated scores to the correct labels, enabling robust outputs even without white-box access. Extensive experiments across classification and QA tasks show substantial reductions in attack success rates while preserving high accuracy on clean data, outperforming state-of-the-art defenses. The approach provides a practical, scalable defense for securing customized LLM APIs in real-world deployments, with code available for reproduction.

Abstract

With the development of customized large language model (LLM) agents, a new threat of black-box backdoor attacks has emerged, where malicious instructions are injected into hidden system prompts. These attacks easily bypass existing defenses that rely on white-box access, posing a serious security challenge. To address this, we propose SLIP, a Soft Label mechanism and key-extraction-guided CoT-based defense against Instruction backdoors in APIs. SLIP is designed based on two key insights. First, to counteract the model's oversensitivity to triggers, we propose a Key-extraction-guided Chain-of-Thought (KCoT). Instead of only considering the single trigger or the input sentence, KCoT prompts the agent to extract task-relevant key phrases. Second, to guide the LLM toward correct answers, our proposed Soft Label Mechanism (SLM) prompts the agent to quantify the semantic correlation between key phrases and candidate answers. Crucially, to mitigate the influence of residual triggers or misleading content in phrases extracted by KCoT, which typically causes anomalous scores, SLM excludes anomalous scores deviating significantly from the mean and subsequently averages the remaining scores to derive a more reliable semantic representation. Extensive experiments on classification and question-answer (QA) tasks demonstrate that SLIP is highly effective, reducing the average attack success rate (ASR) from 90.2% to 25.13% while maintaining high accuracy on clean data and outperforming state-of-the-art defenses. Our code are available in https://github.com/CAU-ISS-Lab/Backdoor-Attack-Defense-LLMs/tree/main/SLIP.

Figures (12)

• Figure 1: (a) Black-box backdoor attack inject backdoor prompt into customized LLMs' system instructions. (b) Previous prompt-based defenses detect poisoned inputs. (c) Proposed SLIP defense bypasses backdoors.
• Figure 2: ASR of Poisoned vs. Clean customized LLMs.
• Figure 3: The correlation scores of trigger. The "red" line is the average correlation score. The target labels are set to 'Negative', 'World', and 'Health Care' (score range is [0,50), [0,25), and [0,16)). More experiments are shown in Appendix \ref{['appendix_pilot_experiments']}.
• Figure 4: Framework of SLIP. $Q_{S\rightarrow \textbf{Y}}$ is correlation-scoring. $S$ and $\textbf{Y}$ are correlation score ranges and label spaces.
• Figure 5: (a) Effectiveness of instance number on GPT-3.5-turbo. (b) Effectiveness of instance type. "S" ("US") is inputs (LLM's understandings of inputs) and $\textbf{E}$ is extracted from "S" ("US").