Automated Test Generation from Program Documentation Encoded in Code Comments

TL;DR

The paper tackles the gap where coverage-driven test generation misses software behaviors documented in code comments and struggles with automated oracles. It introduces JuDoT, a Java test generator that extracts Javadoc-based preconditions and postconditions via JDoc-tor and treats them as explicit test objectives, using a search-based process to produce tests with focal-contract oracles and contract-consistent naming. Empirical evaluation on 118 Java classes shows JuDoT tests 76.6% of postconditions, uncovers 31 unique contract violations, and identifies 45 actual failures, outperforming a code-coverage baseline. The work demonstrates that contract-informed testing can reveal untested behaviors and improve test readability and maintainability, with future work to extend to more languages and more precise contract inferences.

Abstract

Documenting the functionality of software units with code comments, e.g., Javadoc comments, is a common programmer best-practice in software engineering. This paper introduces a novel test generation technique that exploits the code-comment documentation constructively. We originally address those behaviors as test objectives, which we pursue in search-based fashion. We deliver test cases with names and oracles properly contextualized on the target behaviors. Our experiments against a benchmark of 118 Java classes indicate that the proposed approach successfully tests many software behaviors that may remain untested with coverage-driven test generation approaches, and distinctively detects unknown failures.

Figures (4)

• Figure 1: A sample Java class with Javadoc documentation
• Figure 2: Test cases computed with Evo-suite for class GiftPack
• Figure 3: Test cases computed with JuDoT for class GiftPack
• Figure 4: Workflow of JuDoT