Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Test-time Verification via Optimal Transport: Coverage, ROC, & Sub-optimality

Arpan Mukherjee, Marcello Bullo, Debabrota Basu, Deniz Gündüz

TL;DR

This work recasts test-time verification as an optimal-transport problem between a generator's proposal distribution and a verifier-defined target, illuminating how generator coverage, verifier ROC, and sampling strategy jointly shape sub-optimality. It introduces a formal framework with a coverage constraint and a closed-form optimal policy under binary verifier rewards, yielding three distinct regimes for sub-optimality as coverage increases. The authors develop and analyze sequential (AiC, SRS, SMC) and batched (BoN, BRS) sampling algorithms, deriving exact expressions for sub-optimality and computational cost, and validating predictions on Qwen, Llama, and Gemma models. The findings provide precise guidance on when rejection-based versus batch-based resampling is advantageous and how verifier accuracy critically mediates performance, with implications for robust, computation-aware test-time scaling of LLMs.

Abstract

While test-time scaling with verification has shown promise in improving the performance of large language models (LLMs), the role of the verifier and its imperfections remain underexplored. The effect of verification manifests through interactions of three quantities: (i) the generator's coverage, (ii) the verifier's region of convergence (ROC), and (iii) the sampling algorithm's sub-optimality. Though recent studies capture subsets of these factors, a unified framework quantifying the geometry of their interplay is missing. We frame verifiable test-time scaling as a transport problem. This characterizes the interaction of coverage, ROC, and sub-optimality, and uncovers that the sub-optimality--coverage curve exhibits three regimes. A transport regime -- where sub-optimality increases with coverage, a policy improvement regime -- where sub-optimality may decrease with coverage, depending on the verifier's ROC, and a saturation regime -- where sub-optimality plateaus, unaffected by coverage. We further propose and analyze two classes of sampling algorithms -- sequential and batched, and examine how their computational complexities shape these trade-offs. Empirical results with Qwen, Llama, and Gemma models corroborate our theoretical findings.

Test-time Verification via Optimal Transport: Coverage, ROC, & Sub-optimality

TL;DR

This work recasts test-time verification as an optimal-transport problem between a generator's proposal distribution and a verifier-defined target, illuminating how generator coverage, verifier ROC, and sampling strategy jointly shape sub-optimality. It introduces a formal framework with a coverage constraint and a closed-form optimal policy under binary verifier rewards, yielding three distinct regimes for sub-optimality as coverage increases. The authors develop and analyze sequential (AiC, SRS, SMC) and batched (BoN, BRS) sampling algorithms, deriving exact expressions for sub-optimality and computational cost, and validating predictions on Qwen, Llama, and Gemma models. The findings provide precise guidance on when rejection-based versus batch-based resampling is advantageous and how verifier accuracy critically mediates performance, with implications for robust, computation-aware test-time scaling of LLMs.

Abstract

While test-time scaling with verification has shown promise in improving the performance of large language models (LLMs), the role of the verifier and its imperfections remain underexplored. The effect of verification manifests through interactions of three quantities: (i) the generator's coverage, (ii) the verifier's region of convergence (ROC), and (iii) the sampling algorithm's sub-optimality. Though recent studies capture subsets of these factors, a unified framework quantifying the geometry of their interplay is missing. We frame verifiable test-time scaling as a transport problem. This characterizes the interaction of coverage, ROC, and sub-optimality, and uncovers that the sub-optimality--coverage curve exhibits three regimes. A transport regime -- where sub-optimality increases with coverage, a policy improvement regime -- where sub-optimality may decrease with coverage, depending on the verifier's ROC, and a saturation regime -- where sub-optimality plateaus, unaffected by coverage. We further propose and analyze two classes of sampling algorithms -- sequential and batched, and examine how their computational complexities shape these trade-offs. Empirical results with Qwen, Llama, and Gemma models corroborate our theoretical findings.
Paper Structure (38 sections, 15 theorems, 72 equations, 13 figures, 1 table, 4 algorithms)

This paper contains 38 sections, 15 theorems, 72 equations, 13 figures, 1 table, 4 algorithms.

Table of Contents

  1. Motivations & Contributions
  2. Formulation: Test-time Verification as a Transport Plan
  3. Algorithms & Analyses: Sequential and Batched Sampling
  4. Sequential Sampling Algorithms: AiC, SRS, and SMC
  5. Batched Sampling Algorithms: BoN and BRS
  6. Experimental Analysis
  7. Observations. (1) Sub-optimality.
  8. (2) Computational complexity.
  9. (3) Sensitivity to $s_{\rm ver}$.
  10. Discussions and Future Works
  11. Appendix
  12. Literature Review
  13. Approximately correct verifiers.
  14. Algorithm Pseudo-codes
  15. Target-to-proposal Radon-Nikodym Derivative (Proof of Theorem \ref{['theorem: optimal policy']})
  16. ...and 23 more sections

Key Result

Theorem 2.1

For any (prompt, response) pair $(\mathbf{x},\mathbf{y})\in\mathcal{X}\times\mathcal{Y}$, let $r(\mathbf{x},\mathbf{y}) \triangleq \mathds{1}\{\mathbf{y}\in\mathcal{S}_r(\mathbf{x})\}$ be a verifier, where $\mathcal{S}_r(\mathbf{x})\subseteq\mathcal{Y}$. Further, let $\nu_r \triangleq \mathop{\mathr where $m_{\beta}(s) \triangleq s + \sqrt{s(1-s)(\beta-1)}$, and $s_r\triangleq \int_{\mathcal{S}_r}

Figures (13)

  • Figure 1: Regimes of test-time verification.
  • Figure 2: Geometry of test-time verification.
  • Figure 3: Sequential (left) and batched (right) sampling protocols of test-time verification with a generator $G$ and a verifier (light purple box).
  • Figure 4: Sub-optimality (left) and computational complexity (right) as functions of $\beta$ for Qwen3-1.7B and Qwen3-14B. Results are shown for SRS, stochastic SMC, and AiC. Solid green lines denote theoretical predictions as stated in Theorem \ref{['theorem: SRC_SMC_subopt']}. Background shading indicates different coverage regimes, and confidence intervals are shown as shaded bands.
  • Figure 5: Sub-optimality for Qwen3-14B of BRS and BoN with imperfect verifiers (left), and ablation study in $s$ (sensitivity) for SRS, SMC, and AiC (right).
  • ...and 8 more figures

Theorems & Definitions (19)

  • Theorem 2.1: Analytical Form of Optimal Policy
  • Lemma 3.1: Optimal Transport Cost for Hamming Distance
  • Theorem 3.2: Computational Complexity and Sub-optimality of AiC
  • Theorem 3.3: Constraint Violation of AiC
  • Lemma 3.4: Residual Measure
  • Theorem 3.5: Computational Complexity of SRS and SMC
  • Theorem 3.6: Sub-optimality of SRS & SMC
  • Theorem 3.7: Maximum Admissible Batch Size of BoN
  • Theorem 3.8: Sub-optimality of BoN
  • Theorem 3.9: Batch Size of BRS
  • ...and 9 more