LSPRAG: LSP-Guided RAG for Language-Agnostic Real-Time Unit Test Generation

TL;DR

LSPRAG tackles the challenge of real-time unit test generation across multiple programming languages by leveraging Language Server Protocol (LSP) back-ends to provide precise, concise context for large language models. It introduces a three-stage design: (1) key token extraction via lexical LSP data and AST structure to focus on branch-relevant symbols, (2) RAG-based context assembly using symbol and location information with a structured prompt, and (3) a compile-free self-repair loop using LSP diagnostics to fix syntax errors in generated tests. Empirical evaluation across Java, Go, and Python shows substantial improvements in line coverage (up to 213.31% in Java, 174.55% in Go, 31.57% in Python) and significant gains in valid test rates, with the approach maintaining practical latency (~28 seconds per method) and acceptable token costs. The work demonstrates that reusing mature language servers for concise, language-aware context retrieval enables robust, real-time, multi-language test generation, and provides an extensible framework that can be adapted to new languages with modest effort.

Abstract

Automated unit test generation is essential for robust software development, yet existing approaches struggle to generalize across multiple programming languages and operate within real-time development. While Large Language Models (LLMs) offer a promising solution, their ability to generate high coverage test code depends on prompting a concise context of the focal method. Current solutions, such as Retrieval-Augmented Generation, either rely on imprecise similarity-based searches or demand the creation of costly, language-specific static analysis pipelines. To address this gap, we present LSPRAG, a framework for concise-context retrieval tailored for real-time, language-agnostic unit test generation. LSPRAG leverages off-the-shelf Language Server Protocol (LSP) back-ends to supply LLMs with precise symbol definitions and references in real time. By reusing mature LSP servers, LSPRAG provides an LLM with language-aware context retrieval, requiring minimal per-language engineering effort. We evaluated LSPRAG on open-source projects spanning Java, Go, and Python. Compared to the best performance of baselines, LSPRAG increased line coverage by up to 174.55% for Golang, 213.31% for Java, and 31.57% for Python.

Figures (6)

• Figure 1: Unit test generation scenario for focal method checkout. To cover the "true" block of the first branch of the focal method, the generator needs the definition of isValid from another file.
• Figure 2: Integration effort before and after LSP.
• Figure 3: Illustrative example of existing work's context retrieval in unit-test generation. (a) presents a simplified version of the focal method; (b) displays the ground-truth condition context and its location; and (c) compares the context slices and the number of tokens retrieved by three existing RAG techniques.
• Figure 4: Overall workflow of LspRag.
• Figure 5: This figure illustrates the key token selection process. The red box highlights how key tokens are identified. Using the lexical information and CFG, LspRag identifies those tokens involved in or influencing branch decisions.

Theorems & Definitions (8)

• definition 1: Workspace
• definition 2: File
• definition 3: Symbol
• definition 4: Token
• definition 5: Symbol Provider
• definition 6: Token Provider
• definition 7: Definition Provider
• definition 8: Reference Provider