Paraphrasing Adversarial Attack on LLM-as-a-Reviewer

TL;DR

This paper investigates the vulnerability of LLM-based peer review systems to meaning-preserving paraphrasing. It proposes Paraphrasing Adversarial Attack (PAA), a black-box, ICL-guided optimization that paraphrases the abstract to increase review scores while preserving semantics, using $K=8$ paraphrases per step over $T=32$ iterations with similarity and perplexity constraints ($\tau_{\text{sim}}=0.85$, $\alpha_{\text{ppl}}=1.2$). Across five ML/NLP conferences and three LLM reviewers, PAA consistently elevates scores beyond originals and simple paraphrase baselines, with human evaluators confirming semantic integrity and naturalness. The attack reveals self-preference bias and transferability across models, inflates scores relative to human judgments, and yields a detectable rise in review perplexity; paraphrasing defenses partially mitigate the effect, underscoring the need for security evaluations in LLM-assisted peer review.

Abstract

The use of large language models (LLMs) in peer review systems has attracted growing attention, making it essential to examine their potential vulnerabilities. Prior attacks rely on prompt injection, which alters manuscript content and conflates injection susceptibility with evaluation robustness. We propose the Paraphrasing Adversarial Attack (PAA), a black-box optimization method that searches for paraphrased sequences yielding higher review scores while preserving semantic equivalence and linguistic naturalness. PAA leverages in-context learning, using previous paraphrases and their scores to guide candidate generation. Experiments across five ML and NLP conferences with three LLM reviewers and five attacking models show that PAA consistently increases review scores without changing the paper's claims. Human evaluation confirms that generated paraphrases maintain meaning and naturalness. We also find that attacked papers exhibit increased perplexity in reviews, offering a potential detection signal, and that paraphrasing submissions can partially mitigate attacks.

Figures (5)

• Figure 1: Overview of PAA method. The attacker $\mathcal{M}_{\mathrm{atk}}$ generates $K$ paraphrased abstracts using the original abstract and previous paraphrase-score pairs as few-shot examples. Each paraphrased abstract is inserted into the paper and evaluated by LLM-as-a-Reviewer $\mathcal{M}_{\mathrm{rev}}$. This process is repeated for $T$ iterations.
• Figure 2: Attack trajectories showing score improvement over the Original across exploring steps for five attacking models.
• Figure 3: Score difference from Original when attacking LLM-as-a-Reviewer, averaged across five conferences. Matched: the attacking model is the same as the LLM-as-a-Reviewer. Mismatched: they differ.
• Figure 4: Relationship between review score improvement and changes in review content: (a) sentiment score ratio, (b) semantic similarity, and (c) perplexity ratio. The dashed lines indicate linear trends.
• Figure 5: Effect of paraphrasing-based defenses on PAA attack. The y-axis shows the score ratio. The x-axis shows the number of paraphrased paragraphs.