Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering

TL;DR

Satori-SWE introduces Evolutionary Test-Time Scaling (EvoScale) to dramatically improve sample-efficient inference for small language models on real-world SWE tasks. By treating patch generation as an evolutionary process and coupling it with a two-stage mutation-focused SFT plus large-scale RL with potential-based reward shaping, EvoScale guides iterative refinements toward high-quality code patches without relying on costly external verifiers. The approach yields a 32B model whose performance rivals 100B+ models on SWE-bench Verified while using far fewer samples and training data, highlighting practical gains in cost and efficiency. This work demonstrates a viable path to deploying capable SWE assistants in resource-constrained settings and lays groundwork for future agent-based extensions and trajectory-level optimization.

Abstract

Language models (LMs) perform well on standardized coding benchmarks but struggle with real-world software engineering tasks such as resolving GitHub issues in SWE-Bench, especially when model parameters are less than 100B. While smaller models are preferable in practice due to their lower computational cost, improving their performance remains challenging. Existing approaches primarily rely on supervised fine-tuning (SFT) with high-quality data, which is expensive to curate at scale. An alternative is test-time scaling: generating multiple outputs, scoring them using a verifier, and selecting the best one. Although effective, this strategy often requires excessive sampling and costly scoring, limiting its practical application. We propose Evolutionary Test-Time Scaling (EvoScale), a sample-efficient method that treats generation as an evolutionary process. By iteratively refining outputs via selection and mutation, EvoScale shifts the output distribution toward higher-scoring regions, reducing the number of samples needed to find correct solutions. To reduce the overhead from repeatedly sampling and selection, we train the model to self-evolve using reinforcement learning (RL). Rather than relying on external verifiers at inference time, the model learns to self-improve the scores of its own generations across iterations. Evaluated on SWE-Bench-Verified, EvoScale enables our 32B model, Satori-SWE-32B, to match or exceed the performance of models with over 100B parameters while using a few samples. Code, data, and models will be fully open-sourced.

Figures (10)

• Figure 1: Reward score distribution of outputs from a SFT model, with high-scoring outputs concentrated in the long tail.
• Figure 2: Pipeline for SWE Tasks. Given a GitHub issue, the retriever identifies the code files most relevant to the issue. The code editor then generates a code patch to resolve it.
• Figure 3: An Overview of Evolutionary Test-Time Scaling. Given a GitHub issue $x$ and its code context $C(x)$, the editor model $\pi$ first generates a batch of candidate patches $\mathcal{Y}^t$. The reward landscape is illustrated with contour lines, where brighter contours indicate a higher score of a scoring function $R$ (e.g., reward model or unit tests). A set of patches $\mathcal{E}^t$ is selected (e.g., via a scoring function $R$) and combined with $x$ and $C(x)$ to form a conditional prompt (see Section \ref{['subsec:formulation']}), which guides the model to generate the next batch $\mathcal{Y}^{t+1} = \{ y^{t+1}_1, \dots, y^{t+1}_M \}$, increasingly concentrated around the optimum. The process continues under a fixed sampling budget until convergence, after which the final patch is submitted.
• Figure 4: Evolutionary Capability of Different Stages of SFT and RL Models. (a) Reward Model selects the top-$5$ patch candidates from $10$ samples from the previous iteration, and the model iteratively evolves by generating new $10$ samples conditioned on the candidates. Performance of the top-1 sample selected by RM is reported. Without the additional mutation SFT training, the model fails to exhibit evolutionary behavior, even when scaling up the training set. (b) Without RM selection, the model only iteratively evolves by conditioning on 5 random samples from the last iteration. RL training improves the model's initial performance and incentivizes the self-evolution capability, while the SFT model fails to self-evolve without guidance from RM.
• Figure 5: Average Reward Score of Patch Samples at Each Evolution Iteration. Reward scores are normalized via a sigmoid function before average. The SFT model struggles to improve reward scores without the guidance of a reward model to select top-$K$ conditional patch samples, while the RL model consistently self-improves its reward score across iterations without external guidance, validating our theoretical results of monotonic improvement in Section \ref{['subsec:theory']}

Theorems & Definitions (2)

• Definition 1: Myopic Policy
• Proposition 1: Monotonic Improvement