Paper2Rebuttal: A Multi-Agent Framework for Transparent Author Response Assistance

TL;DR

The paper tackles the high-stakes task of author rebuttal in peer review by addressing the grounding, coverage, and coherence challenges of existing methods. It reframes rebuttal writing as a decision-and-evidence organization problem and introduces RebuttalAgent, a multi-agent system that atomizes reviewer concerns, constructs evidence-rich contexts (internal manuscript passages and external briefs), and produces an inspectable response plan before drafting. It also presents RebuttalBench, a benchmark with a rubric evaluating relevance, argumentation, and communication from an author-centric perspective, and demonstrates that the pipeline outperforms direct-to-text baselines across multiple backbones, especially for weaker models. The work emphasizes transparency and author control through a verify-then-write workflow and structured intermediate artifacts, aiming to reduce cognitive burden while maintaining rigorous grounding. The proposed framework, benchmark, and evaluation strategy offer a practical, controllable tool for improving rebuttals in real-world peer-review processes and could influence future tooling for scientific communication.

Abstract

Writing effective rebuttals is a high-stakes task that demands more than linguistic fluency, as it requires precise alignment between reviewer intent and manuscript details. Current solutions typically treat this as a direct-to-text generation problem, suffering from hallucination, overlooked critiques, and a lack of verifiable grounding. To address these limitations, we introduce $\textbf{RebuttalAgent}$, the first multi-agents framework that reframes rebuttal generation as an evidence-centric planning task. Our system decomposes complex feedback into atomic concerns and dynamically constructs hybrid contexts by synthesizing compressed summaries with high-fidelity text while integrating an autonomous and on-demand external search module to resolve concerns requiring outside literature. By generating an inspectable response plan before drafting, $\textbf{RebuttalAgent}$ ensures that every argument is explicitly anchored in internal or external evidence. We validate our approach on the proposed $\textbf{RebuttalBench}$ and demonstrate that our pipeline outperforms strong baselines in coverage, faithfulness, and strategic coherence, offering a transparent and controllable assistant for the peer review process. Code will be released.

Figures (3)

• Figure 1: Overview of our work. Given a manuscript and reviews, (a) direct text generation (SFT on peer-review corpora) often fabricates experiment results and prone to hallucination. (b) Interactive prompting with chat-LLMs depends on manual concern feeding and many iterations. (c) RebuttalAgent reframes rebuttal writing as a decision-and-evidence organization problem, performing concern breakdown, query-conditioned internal and external evidence construction, and strategy-level plan verification with human-in-the-loop checkpoints before drafting the final response.
• Figure 2: Overview of RebuttalAgent. Given a manuscript (PDF) and reviewer comments, the system (1) structures inputs by parsing and compressing the paper with fidelity checks and extracting atomic reviewer concerns with coverage checks; (2) builds concern-conditioned evidence by constructing a query-specific hybrid manuscript context and, when needed, retrieving and summarizing external literature into citation-ready briefs; and (3) generates an inspectable, evidence-linked response plan that is checked for consistency and commitment safety, incorporates optional author feedback, and is then realized into a formal rebuttal draft.
• Figure 3: RebuttalBench statistics and rubric design.(a) Word-cloud and top-word histogram of reviews in RebuttalBench-Corpus, highlighting recurring reviewer emphases (e.g., clarity, novelty, reproducibility). (b) Motivated by these signals, RebuttalBench evaluates rebuttals with a rubric that mirrors these concerns, scoring Relevance, Argumentation Quality, and Communication Quality rather than fluency alone.