Large Language Model Sentinel: LLM Agent for Adversarial Purification
Guang Lin, Toshihisa Tanaka, Qibin Zhao
TL;DR
This paper tackles adversarial attacks on large language models by introducing LLAMOS, a defense framework that purifies adversarial text prior to classification using a dedicated LLM-based defense agent guided by in-context learning. The method consists of two components—Agent Instruction and Defense Guidance—forming a pre-processing purifier that operates without retraining the target LLM, and it demonstrates substantial robustness gains on GLUE across LLAMA-2 and GPT-3.5, including notable reductions in attack success rate and improvements in robust accuracy. The work also explores an online adversarial system with defense and attack agents to study dynamics and generalization, highlighting both the strengths of LLAMOS and the challenges inherent in adversarial purification, such as semantic shifts and potential purification loops. Overall, LLAMOS offers a cost-effective, plug-and-play approach to enhance LLM robustness against unseen textual attacks, with promising directions for leveraging in-context learning and multi-agent interactions in defense strategies.
Abstract
Over the past two years, the use of large language models (LLMs) has advanced rapidly. While these LLMs offer considerable convenience, they also raise security concerns, as LLMs are vulnerable to adversarial attacks by some well-designed textual perturbations. In this paper, we introduce a novel defense technique named Large LAnguage MOdel Sentinel (LLAMOS), which is designed to enhance the adversarial robustness of LLMs by purifying the adversarial textual examples before feeding them into the target LLM. Our method comprises two main components: a) Agent instruction, which can simulate a new agent for adversarial defense, altering minimal characters to maintain the original meaning of the sentence while defending against attacks; b) Defense guidance, which provides strategies for modifying clean or adversarial examples to ensure effective defense and accurate outputs from the target LLMs. Remarkably, the defense agent demonstrates robust defensive capabilities even without learning from adversarial examples. Additionally, we conduct an intriguing adversarial experiment where we develop two agents, one for defense and one for attack, and engage them in mutual confrontation. During the adversarial interactions, neither agent completely beat the other. Extensive experiments on both open-source and closed-source LLMs demonstrate that our method effectively defends against adversarial attacks, thereby enhancing adversarial robustness.