Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Code Review Comprehension: Reviewing Strategies Seen Through Code Comprehension Theories

Pavlína Wurzel Gonçalves, Pooja Rani, Margaret-Anne Storey, Diomidis Spinellis, Alberto Bacchelli

TL;DR

The paper addresses the gap in understanding the cognitive processes behind code review by conducting a qualitative study of 10 experienced reviewers across real-world reviews (25 in total). Using observation, think-aloud protocols, and interviews, it extends Letovsky's code comprehension model into the Code Review Comprehension Model (CRCM), highlighting opportunistic, context-building, and chunking strategies reviewers use to form mental models of changes and evaluate them against expected and ideal solutions. The authors propose the CRCM to explain how reviewers navigate information sources, leverage knowledge bases, and iteratively update mental representations of PRs and the software system, culminating in concrete guidelines for review practices and tool design. The work provides practical implications for designing human-centered review tools and offers a foundation for AI-assisted support in code review, potentially improving efficiency and quality in software development.

Abstract

Despite the popularity and importance of modern code review, the understanding of the cognitive processes that enable reviewers to analyze code and provide meaningful feedback is lacking. To address this gap, we observed and interviewed ten experienced reviewers while they performed 25 code reviews from their review queue. Since comprehending code changes is essential to perform code review and the primary challenge for reviewers, we focused our analysis on this cognitive process. Using Letovsky's model of code comprehension, we performed a theory-driven thematic analysis to investigate how reviewers apply code comprehension to navigate changes and provide feedback. Our findings confirm that code comprehension is fundamental to code review. We extend Letovsky's model to propose the Code Review Comprehension Model and demonstrate that code review, like code comprehension, relies on opportunistic strategies. These strategies typically begin with a context-building phase, followed by code inspection involving code reading, testing, and discussion management. To interpret and evaluate the proposed change, reviewers construct a mental model of the change as an extension of their understanding of the overall software system and contrast mental representations of expected and ideal solutions against the actual implementation. Based on our findings, we discuss how review tools and practices can better support reviewers in employing their strategies and in forming understanding. Data and material: https://doi.org/10.5281/zenodo.14748996

Code Review Comprehension: Reviewing Strategies Seen Through Code Comprehension Theories

TL;DR

The paper addresses the gap in understanding the cognitive processes behind code review by conducting a qualitative study of 10 experienced reviewers across real-world reviews (25 in total). Using observation, think-aloud protocols, and interviews, it extends Letovsky's code comprehension model into the Code Review Comprehension Model (CRCM), highlighting opportunistic, context-building, and chunking strategies reviewers use to form mental models of changes and evaluate them against expected and ideal solutions. The authors propose the CRCM to explain how reviewers navigate information sources, leverage knowledge bases, and iteratively update mental representations of PRs and the software system, culminating in concrete guidelines for review practices and tool design. The work provides practical implications for designing human-centered review tools and offers a foundation for AI-assisted support in code review, potentially improving efficiency and quality in software development.

Abstract

Despite the popularity and importance of modern code review, the understanding of the cognitive processes that enable reviewers to analyze code and provide meaningful feedback is lacking. To address this gap, we observed and interviewed ten experienced reviewers while they performed 25 code reviews from their review queue. Since comprehending code changes is essential to perform code review and the primary challenge for reviewers, we focused our analysis on this cognitive process. Using Letovsky's model of code comprehension, we performed a theory-driven thematic analysis to investigate how reviewers apply code comprehension to navigate changes and provide feedback. Our findings confirm that code comprehension is fundamental to code review. We extend Letovsky's model to propose the Code Review Comprehension Model and demonstrate that code review, like code comprehension, relies on opportunistic strategies. These strategies typically begin with a context-building phase, followed by code inspection involving code reading, testing, and discussion management. To interpret and evaluate the proposed change, reviewers construct a mental model of the change as an extension of their understanding of the overall software system and contrast mental representations of expected and ideal solutions against the actual implementation. Based on our findings, we discuss how review tools and practices can better support reviewers in employing their strategies and in forming understanding. Data and material: https://doi.org/10.5281/zenodo.14748996

Paper Structure

This paper contains 22 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Methodology
  4. Research Questions
  5. Data Collection
  6. Recruitment of Participants
  7. Review&Interview sessions
  8. Transcription
  9. Data Analysis
  10. Ethics and Data Handling
  11. Limitations
  12. Results
  13. RQ1: Scoping Code Review Comprehension
  14. RQ2: Strategies Reviewers Use To Perform Code Review
  15. RQ3: Role of information sources, knowledge base, and mental models
  16. ...and 7 more sections

Figures (4)

  • Figure 1: Letovsky's Model of Code Comprehension letovsky1987cognitive - Code comprehension is an assimilation process using knowledge base and information sources to create a mental model of the code.
  • Figure 2: Coding schema used to structure and code the data according to the selected code comprehension and psychological theories.
  • Figure 3: Code Review Comprehension Model - Code Review Comprehension uses opportunistic strategies to enrich information sources through identifying issues, commenting, and proposing improvements.
  • Figure 4: Code Review Process Strategies