Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A3Test: Assertion-Augmented Automated Test Case Generation

Saranya Alagarsamy, Chakkrit Tantithamthavorn, Aldeida Aleti

TL;DR

The paper targets the challenge of automated test-case generation, noting AthenaTest's limited accuracy due to lacking assertion knowledge and test-signature verification. It introduces A3Test, which augments DL-based test generation with assertion pre-training and a verification module (naming consistency and signatures) using domain adaptation from an assertion-generation task. Evaluation on 5,278 focal methods from Defects4J shows A3Test achieving 40.05% correct test cases and 46.80% focal-method coverage—substantially higher than AthenaTest—while also being significantly faster (97.2% faster overall). Ablation results confirm both assertion pre-training and verification contribute meaningfully to performance, establishing A3Test as a more accurate and efficient approach for generating readable, executable unit tests.

Abstract

Test case generation is an important activity, yet a time-consuming and laborious task. Recently, AthenaTest -- a deep learning approach for generating unit test cases -- is proposed. However, AthenaTest can generate less than one-fifth of the test cases correctly, due to a lack of assertion knowledge and test signature verification. In this paper, we propose A3Test, a DL-based test case generation approach that is augmented by assertion knowledge with a mechanism to verify naming consistency and test signatures. A3Test leverages the domain adaptation principles where the goal is to adapt the existing knowledge from an assertion generation task to the test case generation task. We also introduce a verification approach to verify naming consistency and test signatures. Through an evaluation of 5,278 focal methods from the Defects4j dataset, we find that our A3Test (1) achieves 147% more correct test cases and 15% more method coverage, with a lower number of generated test cases than AthenaTest; (2) still outperforms the existing pre-trained models for the test case generation task; (3) contributes substantially to performance improvement via our own proposed assertion pre-training and the verification components; (4) is 97.2% much faster while being more accurate than AthenaTest.

A3Test: Assertion-Augmented Automated Test Case Generation

TL;DR

The paper targets the challenge of automated test-case generation, noting AthenaTest's limited accuracy due to lacking assertion knowledge and test-signature verification. It introduces A3Test, which augments DL-based test generation with assertion pre-training and a verification module (naming consistency and signatures) using domain adaptation from an assertion-generation task. Evaluation on 5,278 focal methods from Defects4J shows A3Test achieving 40.05% correct test cases and 46.80% focal-method coverage—substantially higher than AthenaTest—while also being significantly faster (97.2% faster overall). Ablation results confirm both assertion pre-training and verification contribute meaningfully to performance, establishing A3Test as a more accurate and efficient approach for generating readable, executable unit tests.

Abstract

Test case generation is an important activity, yet a time-consuming and laborious task. Recently, AthenaTest -- a deep learning approach for generating unit test cases -- is proposed. However, AthenaTest can generate less than one-fifth of the test cases correctly, due to a lack of assertion knowledge and test signature verification. In this paper, we propose A3Test, a DL-based test case generation approach that is augmented by assertion knowledge with a mechanism to verify naming consistency and test signatures. A3Test leverages the domain adaptation principles where the goal is to adapt the existing knowledge from an assertion generation task to the test case generation task. We also introduce a verification approach to verify naming consistency and test signatures. Through an evaluation of 5,278 focal methods from the Defects4j dataset, we find that our A3Test (1) achieves 147% more correct test cases and 15% more method coverage, with a lower number of generated test cases than AthenaTest; (2) still outperforms the existing pre-trained models for the test case generation task; (3) contributes substantially to performance improvement via our own proposed assertion pre-training and the verification components; (4) is 97.2% much faster while being more accurate than AthenaTest.
Paper Structure (18 sections, 1 equation, 2 figures, 7 tables)

This paper contains 18 sections, 1 equation, 2 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background & Related Work
  3. Unit Testing in a Nutshell
  4. DL-based Test Case Generation
  5. A Replication of AthenaTest
  6. A3Test - Assertion Augmented Automated Test Case Generation
  7. Learning Meaningful Assert Statements
  8. Learning Meaningful Test Cases
  9. Generating Test Cases
  10. Verifying Naming Consistency and Test Signatures
  11. EXPERIMENTAL SETUP
  12. Experimental Results
  13. DISCUSSION & FUTURE WORK
  14. Line Coverage Analysis
  15. Generation calls to private fields
  16. ...and 3 more sections

Figures (2)

  • Figure 1: An overview of our A3Test approach, which is a PLBART-based test case generation approach that is augmented by assertion knowledge with a mechanism to verify naming consistency and test signatures.
  • Figure 2: Examples of Test Cases with private field