ChatGPT: Excellent Paper! Accept It. Editor: Imposter Found! Review Rejected

TL;DR

The paper investigates how hidden prompt injections can corrupt AI-assisted peer review by making LLMs overly favorable. It develops a hybrid attack that hides instructions via PDFs, uses topic-shift segments, and iteratively refines prompts, and pairs this with a two-layer defense: structural dual-view reconstruction and behavioural mutation analysis, plus an editor-injected trap to flag LLM-generated reviews. Experiments on a small ICLR 2025 dataset with three LLMs show the attack can raise positive evaluations, while the defense achieves perfect separation between clean and compromised manuscripts. The work exposes the fragility of current peer-review processes under LLM influence and offers a practical, editor-aware framework to restore trust in scientific evaluation.

Abstract

Large Language Models (LLMs) like ChatGPT are now widely used in writing and reviewing scientific papers. While this trend accelerates publication growth and reduces human workload, it also introduces serious risks. Papers written or reviewed by LLMs may lack real novelty, contain fabricated or biased results, or mislead downstream research that others depend on. Such issues can damage reputations, waste resources, and even endanger lives when flawed studies influence medical or safety-critical systems. This research explores both the offensive and defensive sides of this growing threat. On the attack side, we demonstrate how an author can inject hidden prompts inside a PDF that secretly guide or "jailbreak" LLM reviewers into giving overly positive feedback and biased acceptance. On the defense side, we propose an "inject-and-detect" strategy for editors, where invisible trigger prompts are embedded into papers; if a review repeats or reacts to these triggers, it reveals that the review was generated by an LLM, not a human. This method turns prompt injections from vulnerability into a verification tool. We outline our design, expected model behaviors, and ethical safeguards for deployment. The goal is to expose how fragile today's peer-review process becomes under LLM influence and how editorial awareness can help restore trust in scientific evaluation.

Figures (8)

• Figure 1: High level overview of attack pipeline.
• Figure 2: High level overview of defence pipeline.
• Figure 3: Overall rating per paper and model under the prompt-injection attack.
• Figure 4: Average scores for each evaluation dimension across models after the attack.
• Figure 5: Distribution of overall ratings for each model after the attack.