Fixturize: Bridging the Fixture Gap in Test Generation

TL;DR

Fixturize addresses the fixture gap in automated test generation by introducing a three-stage pipeline that first detects fixture dependence, then constructs executable fixture contexts, and finally guides LLMs to synthesize fixture-inclusive unit tests. The authors create FixtureEval, a dedicated benchmark with Python and Java subsets to evaluate fixture dependence classification and fixture-dependent test generation. Empirical results show strong fixture-dependence identification accuracy (88.38%–97.00%) and substantial improvements in Suite Pass rate (18.03%–42.86%), with additional gains in line/branch coverage when Fixturize is integrated with state-of-the-art tools. Overall, the work demonstrates that fixture awareness is a critical, previously missing component for robust, fixture-aware auto-testing pipelines across languages and LLM backbones, and it charts a path for retrieval-augmented and broader-language extensions.

Abstract

Current Large Language Models (LLMs) have advanced automated unit test generation but face a critical limitation: they often neglect to construct the necessary test fixtures, which are the environmental setups required for a test to run. To bridge this gap, this paper proposes Fixturize, a diagnostic framework that proactively identifies fixture-dependent functions and synthesizes test fixtures accordingly through an iterative, feedback-driven process, thereby improving the quality of auto-generated test suites of existing approaches. For rigorous evaluation, the authors introduce FixtureEval, a dedicated benchmark comprising 600 curated functions across two Programming Languages (PLs), i.e., Python and Java, with explicit fixture dependency labels, enabling both the corresponding classification and generation tasks. Empirical results demonstrate that Fixturize is highly effective, achieving 88.38%-97.00% accuracy across benchmarks in identifying the dependence of test fixtures and significantly enhancing the Suite Pass rate (SuitePS) by 18.03%-42.86% on average across both PLs with the auto-generated fixtures. Owing to the maintenance of test fixtures, Fixturize further improves line/branch coverage when integrated with existing testing tools of both LLM-based and Search-based by 16.85%/24.08% and 31.54%/119.66% on average, respectively. The findings establish fixture awareness as an essential, missing component in modern auto-testing pipelines.

Figures (15)

• Figure 1: Illustration of the Fixture Gap and the Solution by Fixturize.
• Figure 2: Workflow of Fixturize
• Figure 3: A Comparative Example of Fixture-Dependent and Fixture-Independent Functions
• Figure 4: A Fixture-Dependent Function Cannot be Called Through Single-Line Invocations
• Figure 5: Prompt Used in the Classification Component