Static Program Analysis Guided LLM Based Unit Test Generation

TL;DR

This work tackles the challenge of generating unit tests for Java methods with large language models by addressing prompt length and context limitations through static program analysis. It proposes a workflow that extracts concise contextual information (focal method signatures, invoked method signatures, and field types) and injects this into LLM prompts, rather than feeding full source files. Across a commercial Java project and Guava, the approach yields more focal methods with generated tests than a full-file-context baseline, while dramatically reducing prompt size and inference time. The findings suggest that precise, language-aware prompt augmentation can substantially improve the practicality and scalability of LLM-based unit test generation, with potential applicability to other languages and future enhancements via in-context learning or fine-tuning.

Abstract

We describe a novel approach to automating unit test generation for Java methods using large language models (LLMs). Existing LLM-based approaches rely on sample usage(s) of the method to test (focal method) and/or provide the entire class of the focal method as input prompt and context. The former approach is often not viable due to the lack of sample usages, especially for newly written focal methods. The latter approach does not scale well enough; the bigger the complexity of the focal method and larger associated class, the harder it is to produce adequate test code (due to factors such as exceeding the prompt and context lengths of the underlying LLM). We show that augmenting prompts with \emph{concise} and \emph{precise} context information obtained by program analysis %of the focal method increases the effectiveness of generating unit test code through LLMs. We validate our approach on a large commercial Java project and a popular open-source Java project.