Auto-GDA: Automatic Domain Adaptation for Efficient Grounding Verification in Retrieval-Augmented Generation
Tobias Leemann, Periklis Petridis, Giuseppe Vietri, Dionysis Manousakas, Aaron Roth, Sergul Aydore
TL;DR
This work addresses domain shift in NLI-based grounding verification for retrieval-augmented generation by introducing Auto-GDA, an unsupervised domain adaptation framework that generates synthetic data, applies label-preserving augmentations, and uses weak supervision from a teacher to tailor lightweight NLI models to realistic RAG inputs. Auto-GDA iteratively generates data with a generator $G$, expands diversity via mutations $M$, and refines labels with a teacher $T$, selecting a top-$K$ subset by minimizing an enhanced distribution-matching objective $L_{tot}$ that blends marginal alignment, label correctness $LDiv$, and a model-utility term $U_f$. Experiments on realistic RAG datasets show that fine-tuning with Auto-GDA data often matches or surpasses the teacher and approaches LLM-level performance while offering about an order of magnitude reduction in inference cost compared to large LLMs. The framework provides a practical, controllable path to efficient grounding verification by domain-adapting compact NLI models without relying on large labeled target sets.
Abstract
While retrieval-augmented generation (RAG) has been shown to enhance factuality of large language model (LLM) outputs, LLMs still suffer from hallucination, generating incorrect or irrelevant information. A common detection strategy involves prompting the LLM again to assess whether its response is grounded in the retrieved evidence, but this approach is costly. Alternatively, lightweight natural language inference (NLI) models for efficient grounding verification can be used at inference time. While existing pre-trained NLI models offer potential solutions, their performance remains subpar compared to larger models on realistic RAG inputs. RAG inputs are more complex than most datasets used for training NLI models and have characteristics specific to the underlying knowledge base, requiring adaptation of the NLI models to a specific target domain. Additionally, the lack of labeled instances in the target domain makes supervised domain adaptation, e.g., through fine-tuning, infeasible. To address these challenges, we introduce Automatic Generative Domain Adaptation (Auto-GDA). Our framework enables unsupervised domain adaptation through synthetic data generation. Unlike previous methods that rely on handcrafted filtering and augmentation strategies, Auto-GDA employs an iterative process to continuously improve the quality of generated samples using weak labels from less efficient teacher models and discrete optimization to select the most promising augmented samples. Experimental results demonstrate the effectiveness of our approach, with models fine-tuned on synthetic data using Auto-GDA often surpassing the performance of the teacher model and reaching the performance level of LLMs at 10% of their computational cost.