Search-based Hyperparameter Tuning for Python Unit Test Generation
Stephan Lukasczyk, Gordon Fraser
TL;DR
This work addresses the often ignored but impactful problem of hyperparameter tuning for SBST by applying differential evolution to tune DynaMOSA and MIO in the Pynguin Python unit-test framework. It conducts a large-scale empirical study contrasting DE with grid search, showing that DE is more resource-efficient and that tuning DynaMOSA substantially improves branch coverage relative to default settings, whereas tuning MIO yields no appreciable gains. The study emphasizes the high cost of tuning but demonstrates potential practical benefit for frequently-run test-generation pipelines, and it discusses broader implications and related tuning approaches. The findings motivate broader adoption of DE-based tuning and point to future work comparing against other tuning frameworks and expanding into parameter-control strategies.
Abstract
Search-based test-generation algorithms have countless configuration options. Users rarely adjust these options and usually stick to the default values, which may not lead to the best possible results. Tuning an algorithm's hyperparameters is a method to find better hyperparameter values, but it typically comes with a high demand of resources. Meta-heuristic search algorithms -- that effectively solve the test-generation problem -- have been proposed as a solution to also efficiently tune parameters. In this work we explore the use of differential evolution as a means for tuning the hyperparameters of the DynaMOSA and MIO many-objective search algorithms as implemented in the Pynguin framework. Our results show that significant improvement of the resulting test suite's coverage is possible with the tuned DynaMOSA algorithm and that differential evolution is more efficient than basic grid search.