MutaGReP: Execution-Free Repository-Grounded Plan Search for Code-Use

TL;DR

MutaGReP tackles the problem of providing code repositories as context to LLM-based code generation without overwhelming the model’s context window. It formulates repo-grounded planning as an execution-free, neural tree search over plans, where each step ties a natural language intent to a small set of code symbols grounded in the target repository. The approach uses two successor-function variants (monotonic and unconstrained), a grounding mechanism that maps intents to top-k symbols via synthetic intents, and scoring and traversal strategies (notably best-first with Likert-based evaluation) to efficiently explore the plan space within a budget. Experiments on LongCodeArena show that tree-searched plans enable strong code-use performance with only a fraction of the repository context, can improve weaker models to match stronger ones, and substantially outperform full-repo-context baselines on hard tasks. This work suggests that structured, repo-grounded planning can provide robust, scalable context for library-style code-use without executing code, with practical implications for accelerators of model performance and interpretability.

Abstract

When a human requests an LLM to complete a coding task using functionality from a large code repository, how do we provide context from the repo to the LLM? One approach is to add the entire repo to the LLM's context window. However, most tasks involve only fraction of symbols from a repo, longer contexts are detrimental to the LLM's reasoning abilities, and context windows are not unlimited. Alternatively, we could emulate the human ability to navigate a large repo, pick out the right functionality, and form a plan to solve the task. We propose MutaGReP (Mutation-guided Grounded Repository Plan Search), an approach to search for plans that decompose a user request into natural language steps grounded in the codebase. MutaGReP performs neural tree search in plan space, exploring by mutating plans and using a symbol retriever for grounding. On the challenging LongCodeArena benchmark, our plans use less than 5% of the 128K context window for GPT-4o but rival the coding performance of GPT-4o with a context window filled with the repo. Plans produced by MutaGReP allow Qwen 2.5 Coder 32B and 72B to match the performance of GPT-4o with full repo context and enable progress on the hardest LongCodeArena tasks. Project page: zaidkhan.me/MutaGReP

Figures (15)

• Figure 1: MutaGReP Overview Given a user request that requires writing code against a specific codebase, we search for realizable plans to solve the user's request using LLM-guided tree search. Our search procedure uses a symbol retriever to constrain search to plans which are implementable with symbols available in the codebase and explores the search space by mutating plans. Each step of the plan consists of a natural language intent and symbols from the codebase that can be used to implement the intent. The user request along with the detailed plan serves as an enriched query that provides necessary context from the codebase to any downstream coding system to convert the plan to code. Our plan search benefits from test-time compute scaling and produces repo-grounded plans without requiring code execution.
• Figure 2: A repo-grounded plan created by MutaGReP on a query from LongCodeArena. Each plan step consists of a natural language intent with top-5 symbols retrieved from the codebase that might be useful for implementing the step.
• Figure 3: Overview of plan search. Each node in the tree is a repo-grounded plan. At every time step, a node is chosen for growing the tree and successors are created by mutating the chosen plan. We use an LLM to implement the successor function.
• Figure 4: Mutation and grounding. The successor function $s$ mutates a plan (left-most column) to generate new plans (right-most column). For each modified intent ($t_2$ and $t_3$), the grounding function $g$ maps the intent to symbols that might be used to implement the intent ($B_2$ and $B_3$).
• Figure 5: Unconstrained mutation outperforms monotonic mutation, especially at lower budgets. Here, we show the symbol recall (% of ground truth symbols in the generated plans) of each mutation strategy using best-first search with the oracle scoring function and branching factor of $3$. This figure also illustrates gains from scaling test-time compute (by increasing budget).