SWE-Spot: Building Small Repo-Experts with Repository-Centric Learning

TL;DR

This paper tackles the problem that small open-weight LLMs struggle to generalize within unfamiliar codebases, a gap TCL cannot close due to limited inference-time knowledge. It introduces Repository-Centric Learning (RCL), emphasizing deep, repository-specific knowledge through four RCX units (Software Design, Contextual Implementation, Evolutionary Replay, Semantic-Runtime Alignment) to train repo-specialized SWE-Spot agents. The authors demonstrate that SWE-Spot-4B, trained with RCL, outperforms much larger open-weight models and matches/ exceeds efficiency-focused commercial APIs across multiple software engineering tasks, while also achieving higher sample efficiency and lower inference costs. They show that RCL yields genuine parametric knowledge rather than mere retrieval improvements and that each RCX unit contributes to cross-task transfer, with ablations confirming synergy among units. The work suggests repository mastery is a distinct, essential dimension for building efficient, privacy-preserving coding agents and provides open-source resources to foster further research in repo-centric software intelligence.

Abstract

The deployment of coding agents in privacy-sensitive and resource-constrained environments drives the demand for capable open-weight Small Language Models (SLMs). However, they suffer from a fundamental capability gap: unlike frontier large models, they lack the inference-time strong generalization to work with complicated, unfamiliar codebases. We identify that the prevailing Task-Centric Learning (TCL) paradigm, which scales exposure across disparate repositories, fails to address this limitation. In response, we propose Repository-Centric Learning (RCL), a paradigm shift that prioritizes vertical repository depth over horizontal task breadth, suggesting SLMs must internalize the "physics" of a target software environment through parametric knowledge acquisition, rather than attempting to recover it via costly inference-time search. Following this new paradigm, we design a four-unit Repository-Centric Experience, transforming static codebases into interactive learning signals, to train SWE-Spot-4B, a family of highly compact models built as repo-specialized experts that breaks established scaling trends, outperforming open-weight models up to larger (e.g., CWM by Meta, Qwen3-Coder-30B) and surpassing/matching efficiency-focused commercial models (e.g., GPT-4.1-mini, GPT-5-nano) across multiple SWE tasks. Further analysis reveals that RCL yields higher training sample efficiency and lower inference costs, emphasizing that for building efficient intelligence, repository mastery is a distinct and necessary dimension that complements general coding capability.

Figures (4)

• Figure 1: Conceptual Illustration on Repository-Centric Learning (RCL) vs. Task-Centric Learning (TCL). TCL scales across repos to learn shared task skills, but struggles on complex codebases due to limited inference-time generalization. RCL instead scales within a repository, diversifying repo-centric experience to enable durable knowledge acquisition and repository mastery.
• Figure 2: A Four-Unit Design of Repository-Centric Experience (RCX)
• Figure 3: Controlled Performance Scaling with Respect to the Amount of Training Samples on Django Issue Resolution. For both the pass rate and the NLL on teacher trajectories, RCL surpasses the peak performance of TCL using significantly less data. RCL uses 10k multi-dimensional mixed samples and trains for 2 epochs, while TCL uses 20k SWE-Smith issue resolving samples and trains for only one epoch.
• Figure 4: Test-time scaling on Django issue resolution. The RCL model consistently outperforms the baselines across all sample budgets, indicating that repository mastery enables foundational improvements orthogonal to test-time scaling.