Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RLCoder: Reinforcement Learning for Repository-Level Code Completion

Yanlin Wang, Yanli Wang, Daya Guo, Jiachi Chen, Ruikai Zhang, Yuchi Ma, Zibin Zheng

TL;DR

Repository-level code completion faces long context and semantic alignment challenges. RLCoder introduces a reinforcement-learning retriever trained without labeled data, guided by a weighted perplexity reward and a stop-signal mechanism, paired with a Split-Aggregate candidate construction to preserve code continuity. Empirical results show state-of-the-art Exact Match gains on CrossCodeEval and RepoEval, strong generalizability across languages, and compatibility with RepoCoder. The framework advances practical repository-level code completion and provides reproducible code and data for further research.

Abstract

Repository-level code completion aims to generate code for unfinished code snippets within the context of a specified repository. Existing approaches mainly rely on retrieval-augmented generation strategies due to limitations in input sequence length. However, traditional lexical-based retrieval methods like BM25 struggle to capture code semantics, while model-based retrieval methods face challenges due to the lack of labeled data for training. Therefore, we propose RLCoder, a novel reinforcement learning framework, which can enable the retriever to learn to retrieve useful content for code completion without the need for labeled data. Specifically, we iteratively evaluate the usefulness of retrieved content based on the perplexity of the target code when provided with the retrieved content as additional context, and provide feedback to update the retriever parameters. This iterative process enables the retriever to learn from its successes and failures, gradually improving its ability to retrieve relevant and high-quality content. Considering that not all situations require information beyond code files and not all retrieved context is helpful for generation, we also introduce a stop signal mechanism, allowing the retriever to decide when to retrieve and which candidates to retain autonomously. Extensive experimental results demonstrate that RLCoder consistently outperforms state-of-the-art methods on CrossCodeEval and RepoEval, achieving 12.2% EM improvement over previous methods. Moreover, experiments show that our framework can generalize across different programming languages and further improve previous methods like RepoCoder. We provide the code and data at https://github.com/DeepSoftwareAnalytics/RLCoder.

RLCoder: Reinforcement Learning for Repository-Level Code Completion

TL;DR

Repository-level code completion faces long context and semantic alignment challenges. RLCoder introduces a reinforcement-learning retriever trained without labeled data, guided by a weighted perplexity reward and a stop-signal mechanism, paired with a Split-Aggregate candidate construction to preserve code continuity. Empirical results show state-of-the-art Exact Match gains on CrossCodeEval and RepoEval, strong generalizability across languages, and compatibility with RepoCoder. The framework advances practical repository-level code completion and provides reproducible code and data for further research.

Abstract

Repository-level code completion aims to generate code for unfinished code snippets within the context of a specified repository. Existing approaches mainly rely on retrieval-augmented generation strategies due to limitations in input sequence length. However, traditional lexical-based retrieval methods like BM25 struggle to capture code semantics, while model-based retrieval methods face challenges due to the lack of labeled data for training. Therefore, we propose RLCoder, a novel reinforcement learning framework, which can enable the retriever to learn to retrieve useful content for code completion without the need for labeled data. Specifically, we iteratively evaluate the usefulness of retrieved content based on the perplexity of the target code when provided with the retrieved content as additional context, and provide feedback to update the retriever parameters. This iterative process enables the retriever to learn from its successes and failures, gradually improving its ability to retrieve relevant and high-quality content. Considering that not all situations require information beyond code files and not all retrieved context is helpful for generation, we also introduce a stop signal mechanism, allowing the retriever to decide when to retrieve and which candidates to retain autonomously. Extensive experimental results demonstrate that RLCoder consistently outperforms state-of-the-art methods on CrossCodeEval and RepoEval, achieving 12.2% EM improvement over previous methods. Moreover, experiments show that our framework can generalize across different programming languages and further improve previous methods like RepoCoder. We provide the code and data at https://github.com/DeepSoftwareAnalytics/RLCoder.
Paper Structure (33 sections, 12 equations, 8 figures, 6 tables, 2 algorithms)

This paper contains 33 sections, 12 equations, 8 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Retrieval-Augmented Generation
  4. Repository-Level Code Completion
  5. Methodology
  6. Overview
  7. Data Construction
  8. Repository Selection
  9. Dependency Analysis
  10. Target Code Selection
  11. Candidate Construction
  12. Reinforcement Learning-based RLRetriever Training
  13. Design of Reward
  14. Stop Signal Mechanism for Candidates Selection
  15. Learning from Reward
  16. ...and 18 more sections

Figures (8)

  • Figure 1: Using fixed window candidates may disrupt the continuity of code semantics, resulting in the definition of functions being split across different code snippets.
  • Figure 2: Due to the limitations of LLMs, inappropriate retrieval can mislead generation, resulting in attempts to call an non-existent attribute.
  • Figure 3: Overview of RLCoder.
  • Figure 4: Data construction pipeline.
  • Figure 5: Candidate construction strategy.
  • ...and 3 more figures