Readability and Understandability Scores for Snippet Assessment: an Exploratory Study
Carlos Eduardo C. Dantas, Marcelo A. Maia
TL;DR
This study addresses how readability and understandability metrics relate to code snippet comprehension and whether they can improve code search or ranking. It uses a four-step design with 30 Java queries, two snippets per query sourced from Google and CROKAGE, and five senior developers providing 60 comprehension ratings, alongside Scalabrino readability and cognitive-complexity-based understandability scores. Results show readability strongly correlates with perceived comprehension (p < 0.01), while understandability is conditionally predictive in specific constructs; inter-rater agreement is low, highlighting subjectivity. The work suggests combining readability with additional features (e.g., writability, legibility) and extending analyses to broader developer populations for more robust code-snippet assessment.
Abstract
Code search engines usually use readability feature to rank code snippets. There are several metrics to calculate this feature, but developers may have different perceptions about readability. Correlation between readability and understandability features has already been proposed, i.e., developers need to read and comprehend the code snippet syntax, but also understand the semantics. This work investigate scores for understandability and readability features, under the perspective of the possible subjective perception of code snippet comprehension. We find that code snippets with higher readability score has better comprehension than lower ones. The understandability score presents better comprehension in specific situations, e.g. nested loops or if-else chains. The developers also mentioned writability aspects as the principal characteristic to evaluate code snippets comprehension. These results provide insights for future works in code comprehension score optimization.