Same Question, Different Words: A Latent Adversarial Framework for Prompt Robustness

TL;DR

Prompt robustness is critical for reliable LLM deployment, but paraphrase-induced performance variation remains a challenge. The authors introduce Latent Adversarial Paraphrasing (LAP), a dual-loop framework that searches for latent continuous paraphrases in hidden space and optimizes the model to perform well on them, using a Lagrangian constraint to preserve semantics. Empirical results across multiple backbones on RobustAlpaca show LAP improves worst-case win-rate by roughly 0.5%–4% absolute over vanilla supervised fine-tuning, without extra paraphrase data or inference-time latency, and it maintains downstream task performance. These findings suggest a practical route to robust prompt handling by shaping embedding geometry, with potential broad impact on real-world LLM deployments.

Abstract

Insensitivity to semantically-preserving variations of prompts (paraphrases) is crucial for reliable behavior and real-world deployment of large language models. However, language models exhibit significant performance degradation when faced with semantically equivalent but differently phrased prompts, and existing solutions either depend on trial-and-error prompt engineering or require computationally expensive inference-time algorithms. In this study, built on the key insight that worst-case prompts exhibit a drift in embedding space, we present Latent Adversarial Paraphrasing (LAP), a dual-loop adversarial framework: the inner loop trains a learnable perturbation to serve as a "latent continuous paraphrase" while preserving semantics through Lagrangian regulation, and the outer loop optimizes the language model parameters on these perturbations. We conduct extensive experiments to demonstrate the effectiveness of LAP across multiple LLM architectures on the RobustAlpaca benchmark with a 0.5%-4% absolution improvement on worst-case win-rate compared with vanilla supervised fine-tuning.

Figures (6)

• Figure 1: Distribution of the ratio between the highest and lowest reward scores (the highest and lowest reward among model response to different paraphrases of user queries) for Llama-2-13b-chat on RobustAlpaca. A higher ratio indicates greater performance variability across semantically equivalent paraphrases.
• Figure 2: The correlation between the $L_2$ worst embedding distance and the performance difference between the original user query and the worst-case paraphrasing (worst distance).The worst distance is correlated with the performance drop caused by the worst-case paraphrasing.
• Figure 3: The distribution of the embedding distance between any two paraphrases of a query (average distance) and the distance between the original query and the worst-case paraphrasing (worst distance). The worst distance is generally larger than the average distance with a drift in their distribution.
• Figure 4: The workflow of our proposed LAP framework consists of an inner loop and an outer loop. For the inner loop, two steps are conducted iteratively to update the perturbation $\delta$ and the Lagrangian multiplier $\lambda$ respectively, while for the outer loop the language model parameter $\theta$ is optimized.
• Figure 5: Training dynamics for the inner-loop optimization with different constraint margin $\epsilon$ on Llama-3-8b backbone.