Double-Anonymous Review for Robotics

TL;DR

This paper surveys the state of double-anonymous review (DAR) versus single-anonymous (SAR) and open review (OR) with a robotics publication lens, highlighting mixed evidence on bias reduction, notably robust status bias and variable gender bias across fields, and documenting substantial implementation and perception challenges. It notes that several robotics venues already employ DAR, while robotics-specific experimental data remain absent, limiting definitive policy guidance. The authors propose concrete next steps, including DAR/SAR comparative studies within major robotics conferences, community surveys of stakeholders, and policy reforms beyond anonymization such as editor representation and reviewer training. The work underscores that perceptions of fairness influence author behavior and policy acceptance, suggesting robotics should pursue targeted, context-aware investigations before broad DAR adoption.

Abstract

Prior research has investigated the benefits and costs of double-anonymous review (DAR, also known as double-blind review) in comparison to single-anonymous review (SAR) and open review (OR). Several review papers have attempted to compile experimental results in peer review research both broadly and in engineering and computer science. This document summarizes prior research in peer review that may inform decisions about the format of peer review in the field of robotics and makes some recommendations for potential next steps for robotics publication.