PRIMG : Efficient LLM-driven Test Generation Using Mutant Prioritization

TL;DR

Mutation testing often yields oversized test suites, limiting practicality. PRIMG tackles this by coupling a mutation-prioritization module, based on a Dynamic Mutant Subsumption Graph ($DMSG$) and $TCAP$, with an LLM-driven test-generation module that iteratively refines tests for syntax and behavior. Empirical results on Code4Arena Solidity projects show that PRIMG can significantly reduce test suite size while preserving mutation coverage, and that prioritization consistently outperforms random mutant selection. The approach offers a scalable, practical workflow for high-quality test generation in smart-contract development and is adaptable to other languages and AI models.

Abstract

Mutation testing is a widely recognized technique for assessing and enhancing the effectiveness of software test suites by introducing deliberate code mutations. However, its application often results in overly large test suites, as developers generate numerous tests to kill specific mutants, increasing computational overhead. This paper introduces PRIMG (Prioritization and Refinement Integrated Mutation-driven Generation), a novel framework for incremental and adaptive test case generation for Solidity smart contracts. PRIMG integrates two core components: a mutation prioritization module, which employs a machine learning model trained on mutant subsumption graphs to predict the usefulness of surviving mutants, and a test case generation module, which utilizes Large Language Models (LLMs) to generate and iteratively refine test cases to achieve syntactic and behavioral correctness. We evaluated PRIMG on real-world Solidity projects from Code4Arena to assess its effectiveness in improving mutation scores and generating high-quality test cases. The experimental results demonstrate that PRIMG significantly reduces test suite size while maintaining high mutation coverage. The prioritization module consistently outperformed random mutant selection, enabling the generation of high-impact tests with reduced computational effort. Furthermore, the refining process enhanced the correctness and utility of LLM-generated tests, addressing their inherent limitations in handling edge cases and complex program logic.

Figures (3)

• Figure 1: The proposed approach for generating a test case using LLMs.
• Figure 2: Overview of the dataset labeling process
• Figure 3: Number of test cases by trial and projects