Outcome-Conditioned Reasoning Distillation for Resolving Software Issues

TL;DR

This work introduces Outcome-Conditioned Reasoning Distillation (O-CRD), a framework that leverages verified historical patches to reconstruct backward, stage-wise reasoning plans for repository-level software issue resolution. By distilling guidance through Stage 1 exemplar mining, Stage 2 exemplar guarding, and Stage 3 backward reasoning distillation (BRD), O-CRD injects structured, outcome-consistent instructions into inference-time workflows without fine-tuning. Across SWE-Bench Lite with multiple base LLMs, O-CRD achieves substantial Pass@1 gains and improved localization, while ablations show the Exemplar Guardian’s critical role and BRD’s superiority over forward search like MCTS under fixed budgets. The approach reduces inference-time cost relative to forward exploration, offering a scalable, practical alternative for powering LLM-based repair pipelines in real-world code repositories.

Abstract

Software issue resolution in large repositories is a long-range decision process: choices made during localization shape the space of viable edits, and missteps can compound into incorrect patches. Despite this, many LLM-based repair pipelines still operate in a reset-and-solve manner, producing fresh reasoning for every new issue instead of carrying forward what worked in past fixes. This is wasteful because repositories routinely contain earlier issues with overlapping structure, failure modes, or constraints, where prior repair experience could provide useful guidance. Existing approaches typically harvest this signal through forward-time trial procedures, such as repeated refinement or search, incurring high inference cost while still risking divergence from the eventual correct patch. We present an Outcome-Conditioned Reasoning Distillation(O-CRD) framework that uses resolved in-repository issues with verified patches as supervision. Starting from a historical fix, the method reconstructs a stage-wise repair trace backward from the verified outcome, then reuses the distilled guidance at inference time to steer file/function localization and patch synthesis, without fine-tuning or online search. On SWE-Bench Lite, this approach increases Pass@1 by 10.4% with GPT-4o, 8.6% with DeepSeek-V3, and 10.3% with GPT-5, indicating that outcome-conditioned reuse of verified repairs can replace costly forward exploration for software issue resolution.

Figures (3)

• Figure 1: O-CRD overview: Stage 1 retrieves in-repository exemplars and selects one via an LLM-based judge. Stage 2 filters candidates into Transferable, Non-transferable, Misleading. Stage 3 distills outcome-conditioned plans $S^*_{file}$, $S^*_{func}$, $S^*_{patch}$ from the exemplar’s ground-truth fix and injects them as in-context guidance during inference.
• Figure 2: Comparison of Pass@1 results on tasks grouped by different difficulty levels.
• Figure 3: SWE-agent official trajectory format sweagent_official.