Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

JustinANN: Realistic Test Generation for Java Programs Driven by Annotations

Baoquan Cui, Rong Qu, Jian Zhang

TL;DR

JustinANN addresses the mismatch between automated test inputs and real-world business scenarios by leveraging annotation-driven constraints to generate realistic test data and assertions for Java programs. By reclassifying Java types into a compact set and designing 7 annotation bases plus 4 combination rules, it enables precise, expressive test-case construction and realistic coverage, implemented atop Soot with reflection. The approach demonstrates partial compatibility with existing annotations, improved boundary-value distribution, and the ability to uncover real defects (26 in open-source projects, with 6 fixed). This annotation-guided framework offers practical benefits for readability, maintainability, and defect discovery in real-world testing pipelines, and it suggests avenues for deeper integration with existing annotation ecosystems and multi-parameter constraints.

Abstract

Automated test case generation is important. However, the automatically generated test input does not always make sense, and the automated assertion is difficult to validate against the program under test. In this paper, we propose JustinANN, a flexible and scalable tool to generate test cases for Java programs, providing realistic test inputs and assertions. We have observed that, in practice, Java programs contain a large number of annotations from programs, which can be considered as part of the user specification. We design a systematic annotation set with 7 kinds of annotations and 4 combination rules based on them to modify complex Java objects. Annotations that modify the fields or return variables of methods can be used to generate assertions that represent the true intent of the program, and the ones that modify the input parameters can be used to generate test inputs that match the real business requirement. We have conducted experiments to evaluate the approach on open source Java programs. The results show that the annotations and their combinations designed in this paper are compatible with existing annotations; our approach is easier to generate test data in, on and outside the boundaries of the requirement domain; and it also helps to find program defects.

JustinANN: Realistic Test Generation for Java Programs Driven by Annotations

TL;DR

JustinANN addresses the mismatch between automated test inputs and real-world business scenarios by leveraging annotation-driven constraints to generate realistic test data and assertions for Java programs. By reclassifying Java types into a compact set and designing 7 annotation bases plus 4 combination rules, it enables precise, expressive test-case construction and realistic coverage, implemented atop Soot with reflection. The approach demonstrates partial compatibility with existing annotations, improved boundary-value distribution, and the ability to uncover real defects (26 in open-source projects, with 6 fixed). This annotation-guided framework offers practical benefits for readability, maintainability, and defect discovery in real-world testing pipelines, and it suggests avenues for deeper integration with existing annotation ecosystems and multi-parameter constraints.

Abstract

Automated test case generation is important. However, the automatically generated test input does not always make sense, and the automated assertion is difficult to validate against the program under test. In this paper, we propose JustinANN, a flexible and scalable tool to generate test cases for Java programs, providing realistic test inputs and assertions. We have observed that, in practice, Java programs contain a large number of annotations from programs, which can be considered as part of the user specification. We design a systematic annotation set with 7 kinds of annotations and 4 combination rules based on them to modify complex Java objects. Annotations that modify the fields or return variables of methods can be used to generate assertions that represent the true intent of the program, and the ones that modify the input parameters can be used to generate test inputs that match the real business requirement. We have conducted experiments to evaluate the approach on open source Java programs. The results show that the annotations and their combinations designed in this paper are compatible with existing annotations; our approach is easier to generate test data in, on and outside the boundaries of the requirement domain; and it also helps to find program defects.
Paper Structure (19 sections, 8 equations, 4 figures, 1 table)

This paper contains 19 sections, 8 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Annotation
  4. Motivation Example
  5. Technique
  6. Type Reclassification
  7. Annotation Basis Design
  8. Annotation Combination
  9. Implementation
  10. Evaluation
  11. Setup
  12. Compatibility
  13. Test Data Distribution
  14. Bug Finding
  15. Threats to Validity
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Overview of Our Approach
  • Figure 2: Annotation Used in Programs
  • Figure 3: Distribution of Test Data Generated by Different Tools. Legends,in,on,out, indicate that the generated test input is inside the given boundary, on the boundary, or outside the boundary.
  • Figure 4: Distribution of Input Data in Test Suites for Each Parameter Constrained by the Annotation. Always In (Out) means that, given a range for an input parameter from the annotation, the test input generated by the tool is always in (out of) the range. Reasonably Distributed means the inputs covers both values in and out of the range.