Generation-Augmented Generation: A Plug-and-Play Framework for Private Knowledge Injection in Large Language Models

TL;DR

Generation-Augmented Generation (GAG) presents a retrieval-free, plug-and-play framework that injects private domain knowledge into a frozen base LLM via a constant-budget one-token interface. It replaces text-level evidence with a representation-level domain expert readout aligned to the base model, enabling reliable selective activation through a prototype-based router (PPR) and scalable multi-domain deployment without base-model updates. Across two private scientific QA benchmarks and mixed-domain settings, GAG achieves substantial specialist gains while preserving open-domain performance and approaching oracle routing limits, addressing key limitations of fine-tuning and retrieval-heavy approaches. The methodology offers governance-friendly, modular knowledge integration suitable for fast-evolving private domains where stability and control over the base model are essential.

Abstract

In domains such as biomedicine, materials, and finance, high-stakes deployment of large language models (LLMs) requires injecting private, domain-specific knowledge that is proprietary, fast-evolving, and under-represented in public pretraining. However, the two dominant paradigms for private knowledge injection each have pronounced drawbacks: fine-tuning is expensive to iterate, and continual updates risk catastrophic forgetting and general-capability regression; retrieval-augmented generation (RAG) keeps the base model intact but is brittle in specialized private corpora due to chunk-induced evidence fragmentation, retrieval drift, and long-context pressure that yields query-dependent prompt inflation. Inspired by how multimodal LLMs align heterogeneous modalities into a shared semantic space, we propose Generation-Augmented Generation (GAG), which treats private expertise as an additional expert modality and injects it via a compact, representation-level interface aligned to the frozen base model, avoiding prompt-time evidence serialization while enabling plug-and-play specialization and scalable multi-domain composition with reliable selective activation. Across two private scientific QA benchmarks (immunology adjuvant and catalytic materials) and mixed-domain evaluations, GAG improves specialist performance over strong RAG baselines by 15.34% and 14.86% on the two benchmarks, respectively, while maintaining performance on six open general benchmarks and enabling near-oracle selective activation for scalable multi-domain deployment.

Figures (14)

• Figure 1: Three paradigms for private knowledge injection. Fine-tuning is expensive and risky; RAG is complex and fragile due to retrieval and long-context pressure. GAG injects private expertise through a constant-budget, modular interface with selective activation.
• Figure 2: GAG overview. A training-free prototype router selects either the general route or one of $N$ plug-and-play domain modules. Each module derives an expert readout from $\mathrm{LLM}_{\text{domain},i}$ and projects it into $\mathrm{LLM}_{\text{base}}$’s embedding space as a single continuous injected token, enabling constant-budget, retrieval-free knowledge injection under a frozen base model.
• Figure 3: GAG two-stage learning.Stage I distills domain competence into $\mathrm{LLM}_{\text{domain},i}$’s internal representations. Stage II learns a lightweight projector $\Pi_i$ that maps the expert readout $\mathbf{k}_i(x)$ into $\mathrm{LLM}_{\text{base}}$’s embedding space, yielding a single continuous injected token for constant-budget, plug-and-play knowledge injection.
• Figure 4: Prototype Plug-and-Play Routing (PPR).Offline: embed historical queries with a frozen encoder $g_\eta$ and cluster them into per-domain prototype banks. Online: route each query by nearest-prototype matching to select the general path or a domain module, enabling training-free, scalable multi-domain expansion without updating the base model.
• Figure 5: Case study (RAG vs. GAG). We contrast RAG’s retrieved evidence and answer with GAG’s injected-token route and answer for the same query and reference. The displayed “Generated Expert Background” is an analysis-only visualization of the injected signal (not exposed in deployment).