Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing LLM-Based Coding Tools through Native Integration of IDE-Derived Static Context

Yichen Li, Yun Peng, Yintong Huo, Michael R. Lyu

TL;DR

The paper tackles the challenge of repo-level code completion with LLMs by exploiting IDE-native static contexts and real-time linting feedback. It introduces IDECoder, a three-phase framework for cross-file context identification, fusion using concise signatures and docstrings, and refinement via as-you-type diagnostics, guided by a chain-of-thought prompting strategy. Preliminary experiments using a Pylance-based PoC and GPT-3.5 demonstrate superior performance over baseline methods in exact match, CodeBLEU, and syntax correctness on cross-file completion tasks. The work suggests a practical path toward more accurate, context-aware code generation within IDEs and lays groundwork for broader extensions to other code-related tasks.

Abstract

Large Language Models (LLMs) have achieved remarkable success in code completion, as evidenced by their essential roles in developing code assistant services such as Copilot. Being trained on in-file contexts, current LLMs are quite effective in completing code for single source files. However, it is challenging for them to conduct repository-level code completion for large software projects that require cross-file information. Existing research on LLM-based repository-level code completion identifies and integrates cross-file contexts, but it suffers from low accuracy and limited context length of LLMs. In this paper, we argue that Integrated Development Environments (IDEs) can provide direct, accurate and real-time cross-file information for repository-level code completion. We propose IDECoder, a practical framework that leverages IDE native static contexts for cross-context construction and diagnosis results for self-refinement. IDECoder utilizes the rich cross-context information available in IDEs to enhance the capabilities of LLMs of repository-level code completion. We conducted preliminary experiments to validate the performance of IDECoder and observed that this synergy represents a promising trend for future exploration.

Enhancing LLM-Based Coding Tools through Native Integration of IDE-Derived Static Context

TL;DR

The paper tackles the challenge of repo-level code completion with LLMs by exploiting IDE-native static contexts and real-time linting feedback. It introduces IDECoder, a three-phase framework for cross-file context identification, fusion using concise signatures and docstrings, and refinement via as-you-type diagnostics, guided by a chain-of-thought prompting strategy. Preliminary experiments using a Pylance-based PoC and GPT-3.5 demonstrate superior performance over baseline methods in exact match, CodeBLEU, and syntax correctness on cross-file completion tasks. The work suggests a practical path toward more accurate, context-aware code generation within IDEs and lays groundwork for broader extensions to other code-related tasks.

Abstract

Large Language Models (LLMs) have achieved remarkable success in code completion, as evidenced by their essential roles in developing code assistant services such as Copilot. Being trained on in-file contexts, current LLMs are quite effective in completing code for single source files. However, it is challenging for them to conduct repository-level code completion for large software projects that require cross-file information. Existing research on LLM-based repository-level code completion identifies and integrates cross-file contexts, but it suffers from low accuracy and limited context length of LLMs. In this paper, we argue that Integrated Development Environments (IDEs) can provide direct, accurate and real-time cross-file information for repository-level code completion. We propose IDECoder, a practical framework that leverages IDE native static contexts for cross-context construction and diagnosis results for self-refinement. IDECoder utilizes the rich cross-context information available in IDEs to enhance the capabilities of LLMs of repository-level code completion. We conducted preliminary experiments to validate the performance of IDECoder and observed that this synergy represents a promising trend for future exploration.
Paper Structure (14 sections, 2 figures, 1 table)

This paper contains 14 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Repo-level Code Completion: Challenges
  3. Identification of Cross-file Contexts
  4. Fusion of Cross-file Contexts
  5. Methodology
  6. Cross-file Context Identification
  7. Cross-file Context Fusion
  8. Linting-based Code Refinement
  9. Preliminary Evaluation
  10. Preliminary Implementation
  11. Evaluation Setup
  12. Evaluation Results
  13. Discussion and Future Plans
  14. ACKNOWLEDGEMENT

Figures (2)

  • Figure 1: An example of the completed code from StarCoder. StarCoder fails to complete a Java program correctly as the in-file context does not provide sufficient type information of Service. To get the service state, the model needs to know the member function getServiceState of Service.
  • Figure 2: The overall framework of IDECoder.