Maximizing Patch Coverage for Testing of Highly-Configurable Software without Exploding Build Times

TL;DR

This paper addresses the challenge of achieving patch coverage in testing highly-configurable software like the Linux kernel, where conventional approaches either incur long build times or fail to cover patch changes. It introduces krepair, an automated method that repairs an existing configuration to cover the lines changed by a patch while preserving most of the original settings, balancing patch coverage and build time through constraint solving and unsat-core-guided repair. The authors implement krepair in Python, integrate constraint extraction from Kconfig, Kbuild, and the C preprocessor, and evaluate it on a statistically significant sample of Linux kernel patches, showing average patch coverage of about $98.5\%$ with only $<1.53\%$ configuration changes in 99% of cases and build times roughly $10.5\times$ faster than the maximal allyesconfig configuration. They also demonstrate that random testing offers far lower patch coverage and higher build costs, and they uncover several build errors that krepair helped reveal, underscoring its practical value for CI in configurable systems. Overall, krepair provides patch-focused CI capability with substantial coverage gains and minimal build-time overhead, enabling more reliable testing of patches in rapidly evolving, configurable software environments.

Abstract

The Linux kernel is highly-configurable, with a build system that takes a configuration file as input and automatically tailors the source code accordingly. Configurability, however, complicates testing, because different configuration options lead to the inclusion of different code fragments. With thousands of patches received per month, Linux kernel maintainers employ extensive automated continuous integration testing. To attempt patch coverage, i.e., taking all changed lines into account, current approaches either use configuration files that maximize total statement coverage or use multiple randomly-generated configuration files, both of which incur high build times without guaranteeing patch coverage. To achieve patch coverage without exploding build times, we propose krepair, which automatically repairs configuration files that are fast-building but have poor patch coverage to achieve high patch coverage with little effect on build times. krepair works by discovering a small set of changes to a configuration file that will ensure patch coverage, preserving most of the original configuration file's settings. Our evaluation shows that, when applied to configuration files with poor patch coverage on a statistically-significant sample of recent Linux kernel patches, krepair achieves nearly complete patch coverage, 98.5% on average, while changing less than 1.53% of the original default configuration file in 99% of patches, which keeps build times 10.5x faster than maximal configuration files.

Figures (6)

• Figure 1: Build system components handling configuration.
• Figure 2: An example patch to the Linux source and the configuration specifications controlling its buildability.
• Figure 3: Workflow of krepair.
• Figure 4: Patch coverage plotted against build times.
• Figure 5: krepair's cached and uncached runtime.