$V_1$: Unifying Generation and Self-Verification for Parallel Reasoners

TL;DR

This work introduces V_1, a framework that unifies generation and verification through efficient pairwise ranking and outperforms recent test-time scaling methods while being significantly more efficient.

Abstract

Test-time scaling for complex reasoning tasks shows that leveraging inference-time compute, by methods such as independently sampling and aggregating multiple solutions, results in significantly better task outcomes. However, a critical bottleneck is verification: sampling is only effective if correct solutions can be reliably identified among candidates. While existing approaches typically evaluate candidates independently via scalar scoring, we demonstrate that models are substantially stronger at pairwise self-verification. Leveraging this insight, we introduce $V_1$, a framework that unifies generation and verification through efficient pairwise ranking. $V_1$ comprises two components: $V_1$-Infer, an uncertainty-guided algorithm using a tournament-based ranking that dynamically allocates self-verification compute to candidate pairs whose relative correctness is most uncertain; and $V_1$-PairRL, an RL framework that jointly trains a single model as both generator and pairwise self-verifier, ensuring the verifier adapts to the generator's evolving distribution. On code generation (LiveCodeBench, CodeContests, SWE-Bench) and math reasoning (AIME, HMMT) benchmarks, $V_1$-Infer improves Pass@1 by up to $10%$ over pointwise verification and outperforms recent test-time scaling methods while being significantly more efficient. Furthermore, $V_1$-PairRL achieves $7$--$9%$ test-time scaling gains over standard RL and pointwise joint training, and improves base Pass@1 by up to 8.7% over standard RL in a code-generation setting.

Figures (16)

• Figure 1: (Left) Pairwise self-verification (using $\textbf{V}_{1}$-Infer, §\ref{['sec:swiss_pairwise_verification']}) outperforms pointwise self-verification in self-verification measured on problems which have both correct and incorrect solutions in their parallel generations (Results with GPT-OSS-20B on LiveCodeBench-V6 prompts). (Middle and Right) Recursive self-aggregation on LiveCodeBench benchmarks shows declining Pass@N (diversity collapse) for both GPT-OSS-20B and Qwen3-4B-Instruct. See §\ref{['sec:why_improve']} for more details.
• Figure 2: Swiss Refinement Overview. Increasing pairwise verifications enables LLMs to better self-verify for selecting the best response among N self-generated solutions. See Section \ref{['sec:swiss_pairwise_verification']}
• Figure 3: Performance after self-verification using $\textbf{V}_{1}$-Infer compared with pointwise self-verification across benchmarks and models at N=16 base generations. Results presented for GPT-OSS-20B and Qwen3-4B-Instruct-2507. Results for GPT-OSS-120B and Qwen3-4B-Thinking-2507 are in Fig. \ref{['fig:appendix-all-verif-bars']} and show similar trends.
• Figure 4: Accuracy vs. total budget (generation + verification calls). Stars, circles, and squares denote $N=8$, $N=16$, and $N=32$ base generations from the LLM, while the total budget $= N + V$ where V is the number of verification calls. $\textbf{V}_{1}$-Infer consistently outperforms pointwise self-verification at equivalent budgets and shows monotonic performance scaling with compute. See Fig. \ref{['fig:appendix_budget_per_benchmark']} for per-benchmark results.
• Figure 5: Comparison with Recursive Self-Aggregation (RSA) venkatraman2025recursiveselfaggregationunlocksdeep on LCB-v6. $\textbf{V}_{1}$-Infer achieves higher accuracy with fewer model calls.