The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched Compute

TL;DR

The paper investigates whether sequential iterative refinement or parallel self-consistency yields better reasoning under matched compute budgets for LLMs. It introduces a training-free inverse-entropy weighted voting (IEW) method and systematically evaluates five open-source models across three challenging benchmarks, demonstrating sequential refinement outperforms parallel approaches in the vast majority of configurations with substantial gains. IEW consistently matches or surpasses baselines, establishing it as a universal aggregation strategy across paradigms. The findings advocate a paradigm shift toward sequential test-time scaling, offering robust performance improvements without fine-tuning and highlighting a practical, compute-efficient path for complex reasoning tasks.

Abstract

We revisit test-time scaling for language model reasoning and ask a fundamental question: at equal token budget and compute, is it better to run multiple independent chains in parallel, or to run fewer chains that iteratively refine through sequential steps? Through comprehensive evaluation across 5 state-of-the-art open source models and 3 challenging reasoning benchmarks, we find that sequential scaling where chains explicitly build upon previous attempts consistently outperforms the dominant parallel self-consistency paradigm in 95.6% of configurations with gains in accuracy upto 46.7%. Further, we introduce inverse-entropy weighted voting, a novel training-free method to further boost the accuracy of sequential scaling. By weighing answers in proportion to the inverse entropy of their reasoning chains, we increase our success rate over parallel majority and establish it as the optimal test-time scaling strategy. Our findings fundamentally challenge the parallel reasoning orthodoxy that has dominated test-time scaling since Wang et al.'s self-consistency decoding (Wang et al., 2022), positioning sequential refinement as the robust default for modern LLM reasoning and necessitating a paradigm shift in how we approach inference-time optimization.

Figures (6)

• Figure 1: Key Results: Qwen3-235B Chain Length Analysis. Sequential reasoning (green) consistently outperforms parallel approaches (red) across AIME-2025 and GPQA-Diamond benchmarks with all chain configurations. Performance numbers are clearly visible, demonstrating advantages up to 46.7% on AIME-2025 and consistent 11-14.6% improvements on GPQA-Diamond, proving the power of iterative refinement.
• Figure 2: Sequential Reasoning Framework Overview. Iterative refinement process where each attempt builds upon previous reasoning, enabling self-correction and verification through progressive steps. The framework demonstrates how sequential chains leverage context accumulation and error correction, culminating in inverse-entropy weighted voting for optimal answer aggregation based on model confidence.
• Figure 3: Creative Task Performance: Sequential vs Parallel Reasoning (using GPT-OSS-120B). Parallel approaches demonstrate superior semantic diversity while sequential approaches show greater lexical diversity across all chain configurations.
• Figure 4: Token Budget Scaling: Sequential vs Parallel Reasoning Laws. Sequential self-refinement (green) consistently outperforms parallel self-consistency (red) across all computational budgets from 2K to 16K tokens, with advantages ranging from 6.6 to 8.9 percentage points. These align with recent wider-vs-deeper inference scaling laws chen2025widerdeeper and suggests sequential methods are more compute-efficient at lower budgets, potentially enabling deployment on edge devices.
• Figure 5: Kimi-K2 Instruct Chain Length Analysis Across All Benchmarks. Sequential reasoning (green) consistently outperforms parallel approaches (red) across AIME-2024, AIME-2025, and GPQA-Diamond with all chain configurations. Notable advantages include 13.3% on AIME-2024 (6 chains), 20.0% on AIME-2025 (6 chains), and consistent 1-3% improvements on GPQA-Diamond, demonstrating robustness across instruction-tuned architectures and diverse reasoning domains.