Dancing in Chains: Strategic Persuasion in Academic Rebuttal via Theory of Mind

TL;DR

This work reframes academic rebuttal as a strategic, theory-of-mind problem and introduces RebuttalAgent, a three-stage ToM-Strategy-Response framework that models reviewer minds, plans rebuttal strategy, and generates evidence-grounded, context-aware responses. It couples a large synthetic dataset, RebuttalBench, with a self-reward reinforcement learning regime and a specialized evaluator, Rebuttal-RM, to achieve scalable, human-aligned persuasion. Empirical results show substantial improvements over strong baselines on automated metrics and human judgments, with ablations confirming the necessity of ToM, strategy planning, and the self-reward design. The work also provides extensive data and methodological details to support reproducibility and discusses ethical considerations for AI-assisted rebuttal in scholarly communication. Overall, it offers a principled, generalizable approach to infusing rebuttals with strategic reasoning and evidence-grounded argumentation, potentially enhancing clarity and constructiveness in peer-review contexts.

Abstract

Although artificial intelligence (AI) has become deeply integrated into various stages of the research workflow and achieved remarkable advancements, academic rebuttal remains a significant and underexplored challenge. This is because rebuttal is a complex process of strategic communication under severe information asymmetry rather than a simple technical debate. Consequently, current approaches struggle as they largely imitate surface-level linguistics, missing the essential element of perspective-taking required for effective persuasion. In this paper, we introduce RebuttalAgent, the first framework to ground academic rebuttal in Theory of Mind (ToM), operationalized through a ToM-Strategy-Response (TSR) pipeline that models reviewer mental state, formulates persuasion strategy, and generates strategy-grounded response. To train our agent, we construct RebuttalBench, a large-scale dataset synthesized via a novel critique-and-refine approach. Our training process consists of two stages, beginning with a supervised fine-tuning phase to equip the agent with ToM-based analysis and strategic planning capabilities, followed by a reinforcement learning phase leveraging the self-reward mechanism for scalable self-improvement. For reliable and efficient automated evaluation, we further develop Rebuttal-RM, a specialized evaluator trained on over 100K samples of multi-source rebuttal data, which achieves scoring consistency with human preferences surpassing powerful judge GPT-4.1. Extensive experiments show RebuttalAgent significantly outperforms the base model by an average of 18.3% on automated metrics, while also outperforming advanced proprietary models across both automated and human evaluations. Disclaimer: the generated rebuttal content is for reference only to inspire authors and assist in drafting. It is not intended to replace the author's own critical analysis and response.

Figures (4)

• Figure 1: Overview of our RebuttalAgent framework. First, we extract each comment from raw reviews and retrieves their relevant context from the paper. Next, based on our TSR pipeline, we collect a tailored strategy and response for each comment, grounded in Theory of Mind. Finally, our RebuttalAgent is trained via Supervised Fine-Tuning, followed by Reinforcement Learning with a self-reward mechanism, enabling both scalability and self-improvement.
• Figure 2: Performance of base models when augmented with the ToM analysis and Strategy generated by our model.
• Figure 3: Heatmap for retrieval effectiveness
• Figure 4: Comparative Evaluation of Model Performance on Rebuttal Quality.