Training-free LLM Verification via Recycling Few-shot Examples

TL;DR

ReFeri addresses the challenge of noisy reasoning in LLMs by introducing a training-free verification framework that recycles provided few-shot examples to both generate and validate multiple candidate outputs. It leverages a Bayes-inspired decomposition of candidate likelihood into forward and backward components, combining them into a final score to select the best response without task-specific training. Across seven benchmarks and three LLMs, ReFeri yields roughly a 4–5 percentage point improvement over strong training-free baselines and demonstrates robust test-time scaling with more candidates and with smaller verification models. The approach offers a practical, domain-agnostic method for reliable LLM output selection, reducing the need for external reward models and supervised tuning while maintaining strong performance.

Abstract

Although LLMs have achieved remarkable performance, the inherent stochasticity of their reasoning process and varying conclusions present significant challenges. Majority voting or Best-of-N with external verification models has been explored to find the most promising solution among multiple LLM outputs. However, these approaches have certain limitations, such as limited applicability or the cost of an additional training step. To address this problem, we propose a novel and effective framework that Recycles Few-shot examples to verify LLM outputs (ReFeri). Our key idea is to additionally utilize the given few-shot examples to evaluate the candidate outputs of the target query, not only using them to generate outputs as the conventional few-shot prompting setup. Specifically, ReFeri evaluates the generated outputs by combining two different scores, designed motivated from Bayes' rule, and subsequently selects the candidate that is both confidently determined and contextually coherent through a few additional LLM inferences. Experiments with three different LLMs and across seven diverse tasks demonstrate that our framework significantly improves the accuracy of LLMs-achieving an average gain of 4.8%-through effective response selection, without additional training.

Figures (18)

• Figure 1: An overview of ReFeri. For $K$ candidate responses from LLMs, ReFeri assigns each candidate a forward confidence score (how likely candidate is to be generated conditioned on few‑shot examples) and a backward consistency score (how well candidate explains the answers of few‑shot examples). Then, the response with the best joint score is selected as the final answer.
• Figure 2: Summary of results. Average accuracy across seven benchmarks with training-free selection methods. ReFeri shows consistent effectiveness (see Section \ref{['sec:4.2']}).
• Figure 2: Ablation study. Evaluation of scoring variants averaged over three generation LLMs, comparing the contribution of metric term (forward and backward) on MATH500 and GPQA.
• Figure 3: Test-time scaling with ReFeri. Accuracy of ReFeri versus other training-free selection methods (Random selection, CoT-WP and USC) on MATH500, GPQA, and MuSR-ta. GPT-4o-mini generate different numbers of candidate responses ($K=1,5,10,15, 20$) using Few-shot CoT.
• Figure 4: Estimation model. Each bar shows the average accuracy of three generation LLMs on MATH500 and GPQA.