UniTSyn: A Large-Scale Dataset Capable of Enhancing the Prowess of Large Language Models for Program Testing

TL;DR

UniTSyn provides a large-scale, multilingual dataset of $2.7$ million focal-test pairs across five major languages, built by static analysis and LSP-based focal extraction to align tests with their focal functions. The authors validate the utility of the dataset by training UniTester, an autoregressive model based on SantaCoder, which achieves notable improvements in test accuracy and coverage over existing code and test-generation models on HumanEval-X. The approach relies on a language-agnostic static analyzer and a scalable pipeline (parsing with tree-sitter, test-function hooks, focal-call analysis, and LSP-based focal extraction) to avoid fragile language-specific heuristics. Overall, UniTSyn demonstrates that paired focal-test data across languages enhances unit test generation and offers a practical, extensible path toward universal ML-assisted software testing.

Abstract

The remarkable capability of large language models (LLMs) in generating high-quality code has drawn increasing attention in the software testing community. However, existing code LLMs often demonstrate unsatisfactory capabilities in generating accurate and complete tests since they were trained on code snippets collected without differentiating between code for testing purposes and other code. In this paper, we present a large-scale dataset UniTSyn, which is capable of enhancing the prowess of LLMs for Unit Test Synthesis. Associating tests with the tested functions is crucial for LLMs to infer the expected behavior and the logic paths to be verified. By leveraging Language Server Protocol, UniTSyn achieves the challenging goal of collecting focal-test pairs without per-project execution setups or per-language heuristics that tend to be fragile and difficult to scale. It contains 2.7 million focal-test pairs across five mainstream programming languages, making it possible to be utilized for enhancing the test generation ability of LLMs. The details of UniTSyn can be found in Table 1. Our experiments demonstrate that, by building an autoregressive model based on UniTSyn, we can achieve significant benefits in learning and understanding unit test representations, resulting in improved generation accuracy and code coverage across all evaluated programming languages. Code and data will be publicly available.

Figures (6)

• Figure 1: UniTSyn workflow overview. We download repositories via the GitHub API and extract their unit tests. The test functions are statically analyzed to identify the call to their focal functions. UniTSyn sends JSON remote procedure calls (RPCs) to language servers to get the location of the focal function definition. Finally, we store the aligned function-level focal-test pairs as training data.
• Figure 2: Test collection phase of UniTSyn frontend. The highlighted words are identifiers that we use to locate testing functions.
• Figure 3: Focal function call analysis phase of UniTSyn frontend. This figure shows a simplified AST of the example test function in \ref{['lst:java_test_func']}. The red node in the AST is the first encountered assertion node with a postorder tree traversal. var_decl is the abbreviation of the variable declaration nodes.
• Figure 4: Prompts used for test generation in different languages
• Figure 5: Effects of training with pairwise test and focal functions. "Baseline" is the SantaCoder model, which didn't train on our collected data. "Unpaired" is the model obtained by training without associating test and focal functions. "Paired" is UniTester trained with focal-test pairs.