Reimagining Peer Review Process Through Multi-Agent Mechanism Design

TL;DR

The paper tackles the software engineering peer-review crisis by reframing it as a mechanism-design problem solvable with engineering methods. It models the research community as a stochastic multi-agent system and advocates applying multi-agent reinforcement learning to redesign incentives. Three interventions are proposed—a persistent Review Credit economy with price dynamics, MARL-based reviewer assignment, and hybrid verification—within a closed-loop architecture and a phased research agenda that incorporates threat models and equity safeguards. If realized, this framework could improve review quality, timeliness, and fairness, enabling sustainable peer review at scale.

Abstract

The software engineering research community faces a systemic crisis: peer review is failing under growing submissions, misaligned incentives, and reviewer fatigue. Community surveys reveal that researchers perceive the process as "broken." This position paper argues that these dysfunctions are mechanism design failures amenable to computational solutions. We propose modeling the research community as a stochastic multi-agent system and applying multi-agent reinforcement learning to design incentive-compatible protocols. We outline three interventions: a credit-based submission economy, MARL-optimized reviewer assignment, and hybrid verification of review consistency. We present threat models, equity considerations, and phased pilot metrics. This vision charts a research agenda toward sustainable peer review.

Figures (1)

• Figure 1: Three-pillar architecture. Note the feedback loop: review quality verification (Pillar 3) directly informs credit issuance and price dynamics (Pillar 1), creating a closed-loop adaptive system.