Awakening Augmented Generation: Learning to Awaken Internal Knowledge of Large Language Models for Question Answering

TL;DR

The paper tackles QA with large language models by addressing limitations of retrieval-augmented and generation-augmented approaches that depend on external data and incur high costs. It introduces Awakening-Augmented Generation (AAG), which combines explicit awakening (a context generator producing concise dummy documents) and implicit awakening (a hypernetwork generating question-specific adapters) with long-context distillation to activate internal knowledge without external resources. Empirically, AAG yields improvements on open-domain and closed-domain QA across NaturalQuestions, TriviaQA, and WebQuestions, with strong OOD generalization and reduced inference cost, outperforming several baselines under similar document conditions. The approach demonstrates that internal knowledge activation can rival or exceed external knowledge augmentation while being more parameter-efficient. This work offers a new perspective on leveraging latent model knowledge to boost QA performance while maintaining efficiency, and points to future extensions including multimodal imagining and broader task applicability.

Abstract

Retrieval-Augmented-Generation and Generation-Augmented-Generation have been proposed to enhance the knowledge required for question answering with Large Language Models (LLMs) by leveraging richer context. However, the former relies on external resources, and both require incorporating explicit documents into the context, which increases execution costs and susceptibility to noise data during inference. Recent works indicate that LLMs model rich knowledge, but it is often not effectively activated and awakened. Inspired by this, we propose a novel knowledge-augmented framework, $\textbf{Awakening-Augmented-Generation}$ (AAG), which mimics the human ability to answer questions using only thinking and recalling to compensate for knowledge gaps, thereby awaking relevant knowledge in LLMs without relying on external resources. AAG consists of two key components for awakening richer context. Explicit awakening fine-tunes a context generator to create a synthetic, compressed document that functions as symbolic context. Implicit awakening utilizes a hypernetwork to generate adapters based on the question and synthetic document, which are inserted into LLMs to serve as parameter context. Experimental results on three datasets demonstrate that AAG exhibits significant advantages in both open-domain and closed-book settings, as well as in out-of-distribution generalization. Our code will be available at \url{https://github.com/Xnhyacinth/IAG}.

Figures (7)

• Figure 1: Compared with RAG and GAG, the proposed AAG eschews external resources, generates a dummy document (explicit awakening) and creates flexible adapters (implicit awakening) for each question.
• Figure 2: Overview of AAG method. In the inference phase, for each question, the explicit awakening (context generator) generates a short dummy document and the implicit awakening (hypernetwork) generates a specific LoRA module. During training, there are two stages: the first stage is the pre-training of the context generator (\ref{['ssec:Compress for Explicit Knowledge']}), aiming at its ability to imagine a short dummy document based on the question, and the second stage is the hypernetwork fine-tuning (\ref{['ssec:Hypernetwork']}) using long context distillation (\ref{['ssec:Knowledge Distillation for contextualized Knowledge']}) to obtain a question-specific LoRA module.
• Figure 3: The Architecture of hypernetwork. Hypernetwork generates LoRA adapter weights for each question. During training, only Hypernetwork, FFN, and Norm weights are updated.
• Figure 4: Zero-Shot results (EM, %) of Llama2-7B on three open-domain QA datasets. The number in parentheses indicates the number of documents used. More zero-shot setting results can be seen in \ref{['sec:zero_sub']}.
• Figure 5: Zero-Shot results (EM, %) of Llama2-13B on three open-domain QA datasets. The number in parentheses indicates the number of documents used.