RTLRepoCoder: Repository-Level RTL Code Completion through the Combination of Fine-Tuning and Retrieval Augmentation

TL;DR

RTLRepoCoder tackles repository-level Verilog code completion by merging long-context, domain-specific fine-tuning with a retrieval-augmented generation (RAG) pipeline. It fine-tunes on open-source Verilog repositories with a 10{,}240-token context and augments long contexts with embedded code snippets retrieved from cross-file contexts, using line-based splitting and long-context embeddings. The approach achieves state-of-the-art performance on the RTL-Repo benchmark, surpassing GPT-4 and larger domain-specific models while remaining parameter-efficient at 6.7B parameters. Comprehensive ablations reveal the contributions of fine-tuning, RAG, embedding choices, splitting strategies, and chunk sizes, providing actionable guidance for deploying repository-scale RTL code completion in real-world hardware design workflows.

Abstract

As an essential part of modern hardware design, manually writing Register Transfer Level (RTL) code such as Verilog is often labor-intensive. Following the tremendous success of large language models (LLMs), researchers have begun to explore utilizing LLMs for generating RTL code. However, current studies primarily focus on generating simple single modules, which can not meet the demands in real world. In fact, due to challenges in managing long-context RTL code and complex cross-file dependencies, existing solutions cannot handle large-scale Verilog repositories in practical hardware development. As the first endeavor to exclusively adapt LLMs for large-scale RTL development, we propose RTLRepoCoder, a groundbreaking solution that incorporates specific fine-tuning and Retrieval-Augmented Generation (RAG) for repository-level Verilog code completion. Open-source Verilog repositories from the real world, along with an extended context size, are used for domain-specific fine-tuning. The optimized RAG system improves the information density of the input context by retrieving relevant code snippets. Tailored optimizations for RAG are carried out, including the embedding model, the cross-file context splitting strategy, and the chunk size. Our solution achieves state-of-the-art performance on public benchmark, significantly surpassing GPT-4 and advanced domain-specific LLMs on Edit Similarity and Exact Match rate. Comprehensive experiments demonstrate the remarkable effectiveness of our approach and offer insights for future work.

Figures (8)

• Figure 1: An illustrative example of repository-level RTL code completion. Due to space limitations, only a portion of the Verilog repository is shown.
• Figure 2: The overall pipeline of RTLRepoCoder
• Figure 3: Edit Similarity of LLMs before and after fine-tuning across samples with different context lengths. The results are presented as percentages(%).
• Figure 4: Two examples showing the prediction changes before and after fine-tuning. The base model outputs irrelevant comment information (a) or prematurely terminate the module (b), while the fine-tuned model generates correct Verilog code.
• Figure 5: Edit Similarity of LLMs with and without RAG on samples exceeding 10k tokens of context. The results are presented as percentages(%).