Are We on the Same Page? Examining Developer Perception Alignment in Open Source Code Reviews

TL;DR

This study investigates how Maintainers and Contributors perceive OSS code reviews, focusing on alignment of objectives, experience of bias, and the role of documentation. Using a mixed-methods design with surveys ($n=289$) and interviews ($n=23$) across $81$ repositories plus repository data, it contrasts the two roles to reveal both shared goals and subtle prioritization differences. Findings show broad agreement on core objectives like Correctness and Quality, but Maintainers emphasize project-direction alignment while Contributors overestimate novelty and undervalue rationale; bias, particularly Familiarity Bias, disproportionately affects newer or underrepresented participants. About $41.18\%$ of respondents witness biased reviews, with types including Familiarity Bias ($65.40\%$) and Approach Difference ($37.02\%$); many misinterpret approach differences as bias, underscoring the need for clearer communication and documentation. The authors propose concrete strategies—enhanced tooling and automation, anonymized or blind reviews, mentor-based onboarding, and AI-assisted review—to reduce delays, improve fairness, and sustain long-term OSS innovation.

Abstract

Code reviews are a critical aspect of open-source software (OSS) development, ensuring quality and fostering collaboration. This study examines perceptions, challenges, and biases in OSS code review processes, focusing on the perspectives of Contributors and Maintainers. Through surveys (n=289), interviews (n=23), and repository analysis (n=81), we identify key areas of alignment and disparity. While both groups share common objectives, differences emerge in priorities, e.g, with Maintainers emphasizing alignment with project goals while Contributors overestimated the value of novelty. Bias, particularly familiarity bias, disproportionately affects underrepresented groups, discouraging participation and limiting community growth. Misinterpretation of approach differences as bias further complicates reviews. Our findings underscore the need for improved documentation, better tools, and automated solutions to address delays and enhance inclusivity. This work provides actionable strategies to promote fairness and sustain the long-term innovation of OSS ecosystems.