Revisiting the Impact of Pursuing Modularity for Code Generation

TL;DR

This work assesses the impact of modularity in code generation by introducing a novel metric for its quantitative measurement and finds that modularity is not a core factor for improving the performance of code generation models.

Abstract

Modular programming, which aims to construct the final program by integrating smaller, independent building blocks, has been regarded as a desirable practice in software development. However, with the rise of recent code generation agents built upon large language models (LLMs), a question emerges: is this traditional practice equally effective for these new tools? In this work, we assess the impact of modularity in code generation by introducing a novel metric for its quantitative measurement. Surprisingly, unlike conventional wisdom on the topic, we find that modularity is not a core factor for improving the performance of code generation models. We also explore potential explanations for why LLMs do not exhibit a preference for modular code compared to non-modular code.

Figures (8)

• Figure 1: In this work, we address the following research question: Given modular and non-modular code snippets with identical functionality, which code type more effectively enhances performance in code generation when used as input for code language models?
• Figure 2: Procedure of computing Cyclomatic Complexity (CC) and Modularity Score ( MoS). We first build control-flow graphs (CFGs) from the given code to derive CC. The CC values are then used to compute MoS as the form of $\text{CC}_{\text{total}}$ and $m^*$.
• Figure 3: Examples of four code categories for the same problem with their modularity scores.
• Figure 4: The prompt template used for converting SC to TMC.
• Figure 5: The prompt template used for two-shot in-context learning with Code Llama.