Position: The AI Conference Peer Review Crisis Demands Author Feedback and Reviewer Rewards

TL;DR

The paper addresses the sustainability crisis in AI conference peer review caused by exploding submission volumes and uneven review quality. It advocates a bi-directional, two-stage review process where authors evaluate review quality and where reviewers receive formal rewards, including a digital badge system and a reviewer impact score to professionalize and incentivize high-quality reviewing. Pragmatic, gradual implementation is emphasized, with considerations for LLM-era challenges and potential gaming, supported by discussions of related work and practical constraints. The proposed reforms aim to realign incentives, improve accountability, and elevate the long-term value of peer review within the AI research ecosystem.

Abstract

The peer review process in major artificial intelligence (AI) conferences faces unprecedented challenges with the surge of paper submissions (exceeding 10,000 submissions per venue), accompanied by growing concerns over review quality and reviewer responsibility. This position paper argues for the need to transform the traditional one-way review system into a bi-directional feedback loop where authors evaluate review quality and reviewers earn formal accreditation, creating an accountability framework that promotes a sustainable, high-quality peer review system. The current review system can be viewed as an interaction between three parties: the authors, reviewers, and system (i.e., conference), where we posit that all three parties share responsibility for the current problems. However, issues with authors can only be addressed through policy enforcement and detection tools, and ethical concerns can only be corrected through self-reflection. As such, this paper focuses on reforming reviewer accountability with systematic rewards through two key mechanisms: (1) a two-stage bi-directional review system that allows authors to evaluate reviews while minimizing retaliatory behavior, (2)a systematic reviewer reward system that incentivizes quality reviewing. We ask for the community's strong interest in these problems and the reforms that are needed to enhance the peer review process.

Figures (7)

• Figure 1: AI Conference Submission Counts. The number of paper submissions to most major AI conferences (e.g. NeurIPS, CVPR, AAAI, ICML, ICLR) exceeded 10,000 by 2025. For example, there was a 59.8% increase in ICLR submissions in 2025 alone. We forecast similar growth in other venues as well.
• Figure 2: Suggested Modification to the Peer Review System. (A) Overview of the standard double-blind peer review system currently adopted by most academic conferences. (B) Our proposed modification to steps two and three of the existing peer review system. Our modification is minimal and it does not disrupts existing timelines, making it easily adaptable to existing systems.
• Figure 3: Reviewer Digital Badge System. We provide an illustrative figure on how digital badges could be displayed on academic profiles (e.g. OpenReview). Digital badges are officially issued by the conferences and reviewers can obtain these badges through our proposed author feedback scores or by meeting specific criteria established by the conference venues.
• Figure 4: Top 20 keywords from years 2018 to 2021 in ICLR Submission
• Figure 5: Top 20 keywords from years 2022 to 2025 in ICLR Submission

Theorems & Definitions (3)

• Definition 1.1: Authors
• Definition 1.2: Reviewers
• Definition 1.3: System