DeepPrune: Parallel Scaling without Inter-trace Redundancy

TL;DR

The paper addresses inefficiency in parallel reasoning for large language models caused by inter-trace redundancy, where most parallel traces converge to the same answer. It proposes DeepPrune, a two-stage framework that trains a specialized judge model with focal loss and oversampling to predict answer equivalence from unfinished traces and uses an online greedy clustering algorithm to prune redundant paths, followed by majority voting. Offline evaluation yields an AUROC of around $0.870$ and strong TNR, while online tests across AIME 2024/25 and GPQA with multiple models show token reductions above $80 ext{%}$ and accuracy within $3$ percentage points of baselines. This approach offers a scalable, efficient solution for parallel reasoning with practical impact on reducing compute without sacrificing solution quality.

Abstract

Parallel scaling has emerged as a powerful paradigm to enhance reasoning capabilities in large language models (LLMs) by generating multiple Chain-of-Thought (CoT) traces simultaneously. However, this approach introduces significant computational inefficiency due to inter-trace redundancy -- our analysis reveals that over 80% of parallel reasoning traces yield identical final answers, representing substantial wasted computation. To address this critical efficiency bottleneck, we propose DeepPrune, a novel framework that enables efficient parallel scaling through dynamic pruning. Our method features a specialized judge model trained with focal loss and oversampling techniques to accurately predict answer equivalence from partial reasoning traces which realizes 0.87 AUROC on equivalence prediction, combined with an online greedy clustering algorithm that dynamically prunes redundant paths while preserving answer diversity. Comprehensive evaluations across three challenging benchmarks (AIME 2024, AIME 2025, and GPQA) and multiple reasoning models demonstrate that DeepPrune achieves remarkable token reduction by over 80% compared to conventional consensus sampling on most cases, while maintaining competitive accuracy within 3 percentage points. Our work establishes a new standard for efficient parallel reasoning, making high-performance reasoning more efficient. Our code and data are here: https://deepprune.github.io/

Figures (4)

• Figure 1: DeepPrune conducts early stopping based on the similarity between reasoning treaces to enhance the efficiency of parallel scaling and save diverse traces.
• Figure 2: Analysis of Inter-trace Redundancy. (a) Distribution of same vs. different answer pairs of reasoning traces, revealing severe redundancy. (b) ROC curve for shallow semantic similarity (SentenceBERT) to distinguish traces with same answers from those with different ones, which shows limited predictive power (AUROC=0.58). (c) ROC curve for LLM-based deep comparison (Qwen3-4B-Instruct) achieves moderate improvement (AUROC=0.66).
• Figure 3: Overview of the DeepPrune framework. The offline training phase (top) involves constructing trace pair datasets with binary labels indicating answer equivalence, then training a judge model using focal loss and oversampling to address class imbalance. The online pruning phase (bottom) leverages the trained judge model to perform dynamic pruning via greedy clustering where traces are assigned to existing clusters or new ones based on similarity predictions, and concludes with majority voting on selected traces to determine the final answer.
• Figure 4: Ablation study on judge model with different truncation strategies for unfinished reasoning traces. We report the classification performance on three reasoning models' trace answer equivalence with different numbers of truncated top tokens (Figure (a)) and different numbers of reasoning words in extracted segments (Figure (b)).