SCOPE: Compress Mathematical Reasoning Steps for Efficient Automated Process Annotation

TL;DR

Process Reward Models (PRMs) enable fine-grained supervision for mathematical reasoning but suffer from expensive per-step annotations. SCOPE introduces a compression-based labeling pipeline that translates reasoning steps into executable code, normalizes steps with Abstract Syntax Trees, and merges equivalent steps into a prefix tree, achieving $O(N)$ annotation complexity. The authors construct a dataset of $196{,}000$ samples with $1.4{,}000{,}000$ step labels using only $5\%$ of prior computational resources, and PRMs trained on this data outperform automated annotation baselines on both Best-of-N and ProcessBench. This work demonstrates a scalable, cost-efficient path to high-quality PRMs for mathematical reasoning with practical implications for AI-assisted problem solving.

Abstract

Process Reward Models (PRMs) have demonstrated promising results in mathematical reasoning, but existing process annotation approaches, whether through human annotations or Monte Carlo simulations, remain computationally expensive. In this paper, we introduce Step COmpression for Process Estimation (SCOPE), a novel compression-based approach that significantly reduces annotation costs. We first translate natural language reasoning steps into code and normalize them through Abstract Syntax Tree, then merge equivalent steps to construct a prefix tree. Unlike simulation-based methods that waste numerous samples on estimation, SCOPE leverages a compression-based prefix tree where each root-to-leaf path serves as a training sample, reducing the complexity from $O(NMK)$ to $O(N)$. We construct a large-scale dataset containing 196K samples with only 5% of the computational resources required by previous methods. Empirical results demonstrate that PRMs trained on our dataset consistently outperform existing automated annotation approaches on both Best-of-N strategy and ProcessBench.

Figures (6)

• Figure 1: Comparison of PRMs training data construction. Simulation-based methods require numerous completions solely for Q-value estimation, with these completions (gray nodes) being discarded without contributing to training. Our compression-based approach eliminates such data waste by merging equivalent steps from all sampled solutions into a prefix tree, where every root-to-leaf path becomes a valuable training instance.
• Figure 2: Illustration of code translation and normalization. The solution of a math problem is first converted into corresponding codes through a code-LLM. Then, we use AST module of Python to derive the abstract syntax tree. Finally, the codes are normalized via their corresponding AST.
• Figure 3: Visualization of prefix tree construction process. Same-colored nodes indicate equivalent normalized step codes. Q-values reflect the proportion of correct solutions passing through each node.
• Figure 4: Distribution of compression rates across different SCOPE variants. Compression Rate = compressed nodes in Trie / raw step count.
• Figure 5: Comparison of time costs (GPU hours) for generating PRM training data across different methods.