SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents

TL;DR

SWE-Prot-Prot-e is introduced, a post-training framework that reframes software repair as an expert-prot-prot-e collaboration problem and combines supervised fine-tuning on expert-augmented trajectories with agentic reinforcement learning that explicitly discourages degenerative looping and unproductive expert collaboration.

Abstract

Small language models (SLMs) offer compelling advantages in cost, latency, and adaptability, but have so far lagged behind larger models on long-horizon software engineering tasks such as SWE-bench, where they suffer from pervasive action looping and low resolution rates. We introduce SWE-Protégé, a post-training framework that reframes software repair as an expert-protégé collaboration problem. In SWE-Protégé, an SLM remains the sole decision-maker while learning to selectively seek guidance from a strong expert model, recognize stalled states, and follow through on expert feedback. Our approach combines supervised fine-tuning on expert-augmented trajectories with agentic reinforcement learning that explicitly discourages degenerative looping and unproductive expert collaboration. We lightly post-train Qwen2.5-Coder-7B-Instruct to achieve 42.4% Pass@1 on SWE-bench Verified, a +25.4% improvement over the prior SLM state of the art, while using expert assistance sparsely (~4 calls per task and 11% of total tokens).

Figures (19)

• Figure 1: (a) Our two-phase pipeline yields SWE-Protégé-7B: Phase uses SFT on expert-augmented trajectories; Phase applies GRPO with trajectory-level rewards. (b) Paraphrased trajectories. Before: the SOTA SLM baseline (SWE-agent-LM-7B) fails to make reliable forward progress and degenerates into unproductive exploration. After Phase : our SLM can invoke the expert, but follow-through on guidance is inconsistent and it often relapses into stalling. After Phase : it learns to escalate when stalled, follow through on guidance, and report back, exhibiting multi-turn pair-programming behavior.
• Figure 2: SWE-Protégé-7B exhibits consistent gains via SFT with increased SWE-smith training data.
• Figure 3: Expert tokens remain consistently low, while total token usage is substantially reduced after Phase II.
• Figure 4: SWE-Protégé substantially reduces per-task cost relative to direct expert execution (the expert solves the entire task). Details in §\ref{['subsec:result-ablation']}.
• Figure 5: Phase II RL (P2) sharply reduces cost/step limit aborts relative to post-SFT (P1) and SWE-smith baselines.