TestSpark: IntelliJ IDEA's Ultimate Test Generation Companion

TL;DR

Test-Spark addresses the challenge of integrating practical, user-friendly unit test generation into daily software development by embedding SBST via EvoSuite and LLM-based generation directly inside IntelliJ IDEA. It introduces a feedback loop that ensures compilable tests, provides rich in-IDE visualization and test manipulation, and offers an extendable architecture for adding new generation methods. The paper details implementation strategies (e.g., Compact Reporter, per-line coverage, configurable LLM prompts) and a comprehensive visualization workflow, along with planned usability studies and preliminary feedback. Its practical impact lies in reducing manual test generation effort while enabling researchers to evaluate and compare different test-generation techniques within real development environments.

Abstract

Writing software tests is laborious and time-consuming. To address this, prior studies introduced various automated test-generation techniques. A well-explored research direction in this field is unit test generation, wherein artificial intelligence (AI) techniques create tests for a method/class under test. While many of these techniques have primarily found applications in a research context, existing tools (e.g., EvoSuite, Randoop, and AthenaTest) are not user-friendly and are tailored to a single technique. This paper introduces TestSpark, a plugin for IntelliJ IDEA that enables users to generate unit tests with only a few clicks directly within their Integrated Development Environment (IDE). Furthermore, TestSpark also allows users to easily modify and run each generated test and integrate them into the project workflow. TestSpark leverages the advances of search-based test generation tools, and it introduces a technique to generate unit tests using Large Language Models (LLMs) by creating a feedback cycle between the IDE and the LLM. Since TestSpark is an open-source (https://github.com/JetBrains-Research/TestSpark), extendable, and well-documented tool, it is possible to add new test generation methods into the plugin with the minimum effort. This paper also explains our future studies related to TestSpark and our preliminary results. Demo video: https://youtu.be/0F4PrxWfiXo

Figures (3)

• Figure 1: Test-Spark workflow
• Figure 2: LLM-based test generation
• Figure 3: Test-Spark's tests visualization