Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EGSS: Entropy-guided Stepwise Scaling for Reliable Software Engineering

Chenhui Mao, Yuanting Lei, Zhixiang Wei, Ming Liang, Zhixiang Wang, Jingxuan Xu, Dajun Chen, Wei Jiang, Yong Li

TL;DR

EGSS addresses two practical bottlenecks in test-time scaling for autonomous software engineering: excessive compute from large ensembles and unreliable patch selection. By leveraging tool-use entropy to dynamically focus exploration at high-uncertainty decision points and by consolidating debugging signals across multiple trajectories into a robust test suite with ensemble voting, EGSS achieves 5–10% higher Resolved% on SWE-Bench-Verified while reducing inference-time tokens by about 28%. The approach yields state-of-the-art performance among open-source LLMs (e.g., GLM-4.6) and demonstrates strong stability across different base models and ensemble sizes. The two-stage framework improves both efficiency and reliability, enhancing practical applicability of ASE systems.

Abstract

Agentic Test-Time Scaling (TTS) has delivered state-of-the-art (SOTA) performance on complex software engineering tasks such as code generation and bug fixing. However, its practical adoption remains limited due to significant computational overhead, primarily driven by two key challenges: (1) the high cost associated with deploying excessively large ensembles, and (2) the lack of a reliable mechanism for selecting the optimal candidate solution, ultimately constraining the performance gains that can be realized. To address these challenges, we propose Entropy-Guided Stepwise Scaling (EGSS), a novel TTS framework that dynamically balances efficiency and effectiveness through entropy-guided adaptive search and robust test-suite augmentation. Extensive experiments on SWE-Bench-Verified demonstrate that EGSS consistently boosts performance by 5-10% across all evaluated models. Specifically, it increases the resolved ratio of Kimi-K2-Intruct from 63.2% to 72.2%, and GLM-4.6 from 65.8% to 74.6%. Furthermore, when paired with GLM-4.6, EGSS achieves a new state-of-the-art among open-source large language models. In addition to these accuracy improvements, EGSS reduces inference-time token usage by over 28% compared to existing TTS methods, achieving simultaneous gains in both effectiveness and computational efficiency.

EGSS: Entropy-guided Stepwise Scaling for Reliable Software Engineering

TL;DR

EGSS addresses two practical bottlenecks in test-time scaling for autonomous software engineering: excessive compute from large ensembles and unreliable patch selection. By leveraging tool-use entropy to dynamically focus exploration at high-uncertainty decision points and by consolidating debugging signals across multiple trajectories into a robust test suite with ensemble voting, EGSS achieves 5–10% higher Resolved% on SWE-Bench-Verified while reducing inference-time tokens by about 28%. The approach yields state-of-the-art performance among open-source LLMs (e.g., GLM-4.6) and demonstrates strong stability across different base models and ensemble sizes. The two-stage framework improves both efficiency and reliability, enhancing practical applicability of ASE systems.

Abstract

Agentic Test-Time Scaling (TTS) has delivered state-of-the-art (SOTA) performance on complex software engineering tasks such as code generation and bug fixing. However, its practical adoption remains limited due to significant computational overhead, primarily driven by two key challenges: (1) the high cost associated with deploying excessively large ensembles, and (2) the lack of a reliable mechanism for selecting the optimal candidate solution, ultimately constraining the performance gains that can be realized. To address these challenges, we propose Entropy-Guided Stepwise Scaling (EGSS), a novel TTS framework that dynamically balances efficiency and effectiveness through entropy-guided adaptive search and robust test-suite augmentation. Extensive experiments on SWE-Bench-Verified demonstrate that EGSS consistently boosts performance by 5-10% across all evaluated models. Specifically, it increases the resolved ratio of Kimi-K2-Intruct from 63.2% to 72.2%, and GLM-4.6 from 65.8% to 74.6%. Furthermore, when paired with GLM-4.6, EGSS achieves a new state-of-the-art among open-source large language models. In addition to these accuracy improvements, EGSS reduces inference-time token usage by over 28% compared to existing TTS methods, achieving simultaneous gains in both effectiveness and computational efficiency.
Paper Structure (31 sections, 4 equations, 6 figures, 3 tables, 1 algorithm)

This paper contains 31 sections, 4 equations, 6 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Scaling Paradigms and Search Efficiency
  4. Reliability in Patch Selection
  5. Preliminary
  6. Tool Use Entropy Heterogeneity
  7. Self-Deceptive Debugging
  8. Problem Formulation
  9. Method
  10. Overview
  11. Dynamic Stepwise Search
  12. Test Consolidation Augmentation
  13. Debugging action extraction.
  14. Consolidated test suite generation.
  15. Patch evaluation and filtering.
  16. ...and 16 more sections

Figures (6)

  • Figure 1: Performance and token usage of popular test-time scaling methods compared with entropy-guided stepwise scaling on SWE-Bench Verified, using Kimi-K2-Instruct as the base model.
  • Figure 2: Tool entropy distribution along agent trajectories in SWE-Bench cases
  • Figure 3: Trajectory-Aware Analysis of Debugging Processes in Autonomous Agents on SWE-Bench-Verified, Using Kimi-K2-Instruct as the base model
  • Figure 4: Overview of Entropy-guided Stepwise Scaling
  • Figure 5: Average token usage per instance on the SWE-Bench benchmark, aggregated across different ensemble sizes and sampling strategies for Kimi-K2-Instruct and GLM-4.6.
  • ...and 1 more figures