Towards A Sustainable Future for Peer Review in Software Engineering
Esteban Parra, Sonia Haiduc, Preetha Chatterjee, Ramtin Ehsani, Polina Iaremchuk
TL;DR
The paper addresses the unsustainability of current peer review in software engineering due to rising submissions and insufficient reviewer capacity. It proposes a multi-faceted approach focusing on scalable training for junior reviewers, responsible integration of AI tools, and redesigned incentives to broaden participation and maintain review quality. Key contributions include a scalable online training framework, policy guidance for AI use in reviewing, and incentive mechanisms such as reviewer badges and a new newbie award. The proposed framework aims to preserve rigorous peer evaluation while enabling growth and inclusivity in SE research, with practical steps for implementation and policy alignment with major professional organizations.
Abstract
Peer review is the main mechanism by which the software engineering community assesses the quality of scientific results. However, the rapid growth of paper submissions in software engineering venues has outpaced the availability of qualified reviewers, creating a growing imbalance that risks constraining and negatively impacting the long-term growth of the Software Engineering (SE) research community. Our vision of the Future of the SE research landscape involves a more scalable, inclusive, and resilient peer review process that incorporates additional mechanisms for: 1) attracting and training newcomers to serve as high-quality reviewers, 2) incentivizing more community members to serve as peer reviewers, and 3) cautiously integrating AI tools to support a high-quality review process.