ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning

TL;DR

This work proposes ReflexiCoder, a novel reinforcement learning (RL) framework that internalizes the structured reasoning trajectory, encompassing initial generation, bug and optimization aware reflection, and self-correction, directly into the model's weights, directly into the model's weights.

Abstract

While Large Language Models (LLMs) have revolutionized code generation, standard "System 1" approaches, generating solutions in a single forward pass, often hit a performance ceiling when faced with complex algorithmic tasks. Existing iterative refinement strategies attempt to bridge this gap at inference time, yet they predominantly rely on external oracles, execution feedback, or computationally expensive prompt-response cycles. In this work, we propose ReflexiCoder, a novel reinforcement learning (RL) framework that internalizes the structured reasoning trajectory, encompassing initial generation, bug and optimization aware reflection, and self-correction, directly into the model's weights. Unlike prior methods, ReflexiCoder shifts the paradigm from external-dependent refinement to an intrinsic, fully autonomous self-reflection and self-correction capabilities at inference time. We utilize an RL-zero training paradigm with granular reward functions to optimize the entire reflection-correction trajectory, teaching the model how to debug without reliance on ground-truth feedback or execution engines at inference time. Extensive experiments across seven benchmarks demonstrate that our ReflexiCoder-8B establishes a new state-of-the-art (SOTA) among leading open-source models in the 1.5B-14B range, achieving 94.51% (87.20%) on HumanEval (Plus), 81.80% (78.57%) on MBPP (Plus), 35.00% on BigCodeBench, 52.21% on LiveCodeBench, and 37.34% on CodeForces in a single-attempt setting, rivaling or surpassing proprietary models like GPT-5.1. Notably, our framework is significantly more token-efficient than base models, reducing inference-time compute overhead by approximately 40% through disciplined, high-speed reasoning and reflection patterns. Source code is available at https://github.com/juyongjiang/ReflexiCoder.

Figures (10)

• Figure 1: A comparative overview of iterative code refinement workflows at inference time. (a) Existing code LLMs often struggle to generate correct solutions for complex programming tasks on a single attempt. (b) Prior practices mitigate this by relying on external feedback (e.g., compilers, reflection or human oracles). (c) Our proposed ReflexiCoder fosters an intrinsic capability to self-reflect and self-correct via a structured reasoning trajectory, eliminating the need for external oracles and environmental interaction.
• Figure 2: The architecture of ReflexiCoder formulates code generation as an RL-optimized intrinsic self-debugging trajectory. A carefully designed composite reward jointly incentivizes reflection quality and correction success.
• Figure 3: Analysis of the impact of reasoning and reflection. We compare ReflexiCoder against baselines that lack a structured reasoning and reflection step. Performance is pass@1 (%). The significant performance gap highlights that the structured reasoning-reflection cycle is the key driver of improvement.
• Figure 4: Scaling analysis of ReflexiCoder with model size. Performance is pass@1 (%) across four representative benchmarks. The performance grow with model scale, indicating a super-linear benefit.
• Figure 5: Pass@1 sensitivity to reward weights Format ($\xi$), Iter. Quality ($\varphi$), Efficiency ($\psi$) on four benchmarks.