Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RAL2M: Retrieval Augmented Learning-To-Match Against Hallucination in Compliance-Guaranteed Service Systems

Mengze Hong, Di Jiang, Jiangtao Wen, Zhiyang Su, Yawen Li, Yanjie Sun, Guan Wang, Chen Jason Zhang

TL;DR

This work tackles hallucination in compliance-critical service systems by grounding responses in retrieved knowledge rather than generation. It introduces Retrieval-Augmented Learning-to-Match (RAL2M), repositioning LLMs as query-response matching judges and employing a query-adaptive latent ensemble to calibrate cross-model judgments. The framework uses an energy-based model with three potentials to jointly reason about query context, judge competence, and inter-model interaction, enabling zero-generation decisions when no good match exists. Empirical results on a large multi-domain dataset show that the latent ensemble achieves higher accuracy and substantially lower hallucination rates than baselines, highlighting practical gains for safe, knowledge-grounded services. The work also discusses data- and computation-related trade-offs and points to future directions in leveraging latent representations for more robust, domain-adaptive QA systems.

Abstract

Hallucination is a major concern in LLM-driven service systems, necessitating explicit knowledge grounding for compliance-guaranteed responses. In this paper, we introduce Retrieval-Augmented Learning-to-Match (RAL2M), a novel framework that eliminates generation hallucination by repositioning LLMs as query-response matching judges within a retrieval-based system, providing a robust alternative to purely generative approaches. To further mitigate judgment hallucination, we propose a query-adaptive latent ensemble strategy that explicitly models heterogeneous model competence and interdependencies among LLMs, deriving a calibrated consensus decision. Extensive experiments on large-scale benchmarks demonstrate that the proposed method effectively leverages the "wisdom of the crowd" and significantly outperforms strong baselines. Finally, we discuss best practices and promising directions for further exploiting latent representations in future work.

RAL2M: Retrieval Augmented Learning-To-Match Against Hallucination in Compliance-Guaranteed Service Systems

TL;DR

This work tackles hallucination in compliance-critical service systems by grounding responses in retrieved knowledge rather than generation. It introduces Retrieval-Augmented Learning-to-Match (RAL2M), repositioning LLMs as query-response matching judges and employing a query-adaptive latent ensemble to calibrate cross-model judgments. The framework uses an energy-based model with three potentials to jointly reason about query context, judge competence, and inter-model interaction, enabling zero-generation decisions when no good match exists. Empirical results on a large multi-domain dataset show that the latent ensemble achieves higher accuracy and substantially lower hallucination rates than baselines, highlighting practical gains for safe, knowledge-grounded services. The work also discusses data- and computation-related trade-offs and points to future directions in leveraging latent representations for more robust, domain-adaptive QA systems.

Abstract

Hallucination is a major concern in LLM-driven service systems, necessitating explicit knowledge grounding for compliance-guaranteed responses. In this paper, we introduce Retrieval-Augmented Learning-to-Match (RAL2M), a novel framework that eliminates generation hallucination by repositioning LLMs as query-response matching judges within a retrieval-based system, providing a robust alternative to purely generative approaches. To further mitigate judgment hallucination, we propose a query-adaptive latent ensemble strategy that explicitly models heterogeneous model competence and interdependencies among LLMs, deriving a calibrated consensus decision. Extensive experiments on large-scale benchmarks demonstrate that the proposed method effectively leverages the "wisdom of the crowd" and significantly outperforms strong baselines. Finally, we discuss best practices and promising directions for further exploiting latent representations in future work.
Paper Structure (45 sections, 19 equations, 5 figures, 10 tables, 1 algorithm)

This paper contains 45 sections, 19 equations, 5 figures, 10 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Learning-to-Match
  4. LLM Ensemble
  5. Preliminaries
  6. Problem Formulation
  7. The LLM Heterogeneity Dilemma
  8. Proposed Methods
  9. Model Architecture
  10. Model Parameterization
  11. Query-conditioned Context Potential $\Phi(\mathbf{Z}; \mathbf{e}_q)$.
  12. Compatibility Potential $\Lambda(\mathbf{s}, \mathbf{Z})$.
  13. Interaction Potential $\Psi(y, \mathbf{Z}; \mathbf{e}_q)$.
  14. Posterior Approximation
  15. Training and Inference
  16. ...and 30 more sections

Figures (5)

  • Figure 1: System comparison under cautionary queries: generative system (left) produces hallucinated responses; retrieval system with LLM judgment (right) returns verified and appropriate response.
  • Figure 2: Model architecture of query-adaptive latent ensemble with query embedding $\mathbf{e}_q$ and prediction $y$.
  • Figure 3: Comparison of accuracy and hallucination rate across datasets.
  • Figure 4: Distribution of the number of correct judges.
  • Figure 5: Convergence behaviour of the proposed latent ensemble model.