Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Efficient Rubric-based Generative Verifier for Search-Augmented LLMs

Linyue Ma, Yilong Xu, Xiang Long, Zhi Zheng

TL;DR

This work introduces nugget-as-rubric as a unified, verifiable reward paradigm for both short- and long-form search-augmented LLMs. It couples an automatic rubrics-construction pipeline with a 4B rubric verifier, Search-Gen-V, trained via distillation in a two-stage SFT/RL process to produce robust, scalable rewards. Across short-form and long-form tasks, Search-Gen-V demonstrates competitive accuracy with much larger verifiers and enables efficient reward computation for RL in retrieval-enhanced generation. The approach improves verifiability, robustness, and throughput, with practical implications for deploying scalable, trustworthy search-augmented LLM systems. Limitations include rubrics-construction time and the current absence of full RL integration in the reported experiments, guiding future work toward faster rubrics synthesis and end-to-end RL evaluation.

Abstract

Search augmentation empowers Large Language Models with retrieval capabilities to overcome the limitations imposed by static parameters. Recently, Reinforcement Learning leverages tailored reward signals as a viable technique to enhance LLMs performing tasks involving search. However, existing reward modeling for search-augmented LLMs faces several limitations. Rule-based rewards, such as Exact Match, are verifiable but fragile to variations in expression and cannot be applied to long-form workloads. In contrast, generative rewards improve robustness, but designing verifiable and stable rewards for long-form workloads in dynamic corpora remains challenging and also incurs high computational costs. In this paper, we propose a unified and verifiable paradigm, "nugget-as-rubric", which treats atomic information points as structured evaluation criteria for different search-augmentation workloads. Short-form tasks correspond to a single rubric, whereas long-form tasks expand to multiple rubrics aligned with the question's information needs. To support long-form settings, we design an automatic rubric construction pipeline based on query rewriting, which can automatically retrieve passages relevant to each question and extract rubrics from them, both from static corpora and from dynamic online web content. Furthermore, we introduce \textbf{Search-Gen-V}, a 4B-parameter efficient generative verifier under our proposed verifiable paradigm, which is trained via the idea of distillation and a two-stage strategy. Experimental results show that Search-Gen-V achieves strong verification accuracy across different workloads, making it a scalable, robust, and efficient verifiable reward constructor for search-augmented LLMs.

An Efficient Rubric-based Generative Verifier for Search-Augmented LLMs

TL;DR

This work introduces nugget-as-rubric as a unified, verifiable reward paradigm for both short- and long-form search-augmented LLMs. It couples an automatic rubrics-construction pipeline with a 4B rubric verifier, Search-Gen-V, trained via distillation in a two-stage SFT/RL process to produce robust, scalable rewards. Across short-form and long-form tasks, Search-Gen-V demonstrates competitive accuracy with much larger verifiers and enables efficient reward computation for RL in retrieval-enhanced generation. The approach improves verifiability, robustness, and throughput, with practical implications for deploying scalable, trustworthy search-augmented LLM systems. Limitations include rubrics-construction time and the current absence of full RL integration in the reported experiments, guiding future work toward faster rubrics synthesis and end-to-end RL evaluation.

Abstract

Search augmentation empowers Large Language Models with retrieval capabilities to overcome the limitations imposed by static parameters. Recently, Reinforcement Learning leverages tailored reward signals as a viable technique to enhance LLMs performing tasks involving search. However, existing reward modeling for search-augmented LLMs faces several limitations. Rule-based rewards, such as Exact Match, are verifiable but fragile to variations in expression and cannot be applied to long-form workloads. In contrast, generative rewards improve robustness, but designing verifiable and stable rewards for long-form workloads in dynamic corpora remains challenging and also incurs high computational costs. In this paper, we propose a unified and verifiable paradigm, "nugget-as-rubric", which treats atomic information points as structured evaluation criteria for different search-augmentation workloads. Short-form tasks correspond to a single rubric, whereas long-form tasks expand to multiple rubrics aligned with the question's information needs. To support long-form settings, we design an automatic rubric construction pipeline based on query rewriting, which can automatically retrieve passages relevant to each question and extract rubrics from them, both from static corpora and from dynamic online web content. Furthermore, we introduce \textbf{Search-Gen-V}, a 4B-parameter efficient generative verifier under our proposed verifiable paradigm, which is trained via the idea of distillation and a two-stage strategy. Experimental results show that Search-Gen-V achieves strong verification accuracy across different workloads, making it a scalable, robust, and efficient verifiable reward constructor for search-augmented LLMs.
Paper Structure (32 sections, 8 equations, 6 figures, 4 tables, 1 algorithm)

This paper contains 32 sections, 8 equations, 6 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Search-augmented LLMs.
  4. Reinforcement Learning for Search.
  5. Methodology
  6. Nugget-as-Rubric: Definitions from a Unified Perspective
  7. Automatic Rubrics Construction
  8. Search-Gen-V: An Efficient Rubric Verifier
  9. Experiments
  10. Experimental Setup
  11. Implementation Details.
  12. Workloads and Baselines.
  13. Validation Result
  14. Long-form Workload Result
  15. Short-form Workload Result
  16. ...and 17 more sections

Figures (6)

  • Figure 1: Two typical workloads for search-augmented LLMs. Existing reward modeling methods suffer from issues in robustness, verifiability, and computational cost. Our approach in this work not only unifies both types of workloads but also achieves better verifiability and higher efficiency.
  • Figure 2: Our Search-Gen-V achieves a favorable balance between efficiency and performance in verifying rubrics for long-form answers.
  • Figure 3: Illustration of the pipeline of our rubric-based verifiable reward modeling, which consists of two parts. Left (§ \ref{['sec:automatic_rubrics_construction']}): automated generation of nugget-based rubrics. Right (§ \ref{['sec:search_gen_v']}): rubric verification using Search-Gen-V, which ultimately produces the reward.
  • Figure 4: Evaluation results of the long-form workload, DeepResearch Bench. Left figures: Rubric-based scores are generated by Search-Gen-V-4B. r denotes the Pearson correlation coefficient, and p indicates statistical significance. Right table: Accuracy comparison on verifying rubrics in long-form answers from DeepResearch Bench. All other settings are the same as in Table \ref{['tb:validation']}.
  • Figure 5: Results of short-form workload, evaluating on HotpotQA. Left: cases of verifying rubrics satisfaction, where EM misjudges all cases due to bad robustness, while Search-Gen-V provides correct labels. Right: comparison of judgment accuracy on the 585 samples misjudged by EM.
  • ...and 1 more figures