Self-Tuning: Instructing LLMs to Effectively Acquire New Knowledge through Self-Teaching

TL;DR

Self-Tuning presents a three-stage framework for teaching LLMs to acquire new knowledge from unseen raw documents, inspired by the Feynman Technique. A novel Self-Teaching strategy decomposes learning into memorization, comprehension, and self-reflection, with tasks designed to be self-supervised and document-driven. The Wiki-Newpages-2023-QA datasets enable rigorous evaluation of memorization, extraction, and reasoning across single-, multi-, and cross-domain settings. Empirical results across multiple models show that Self-Tuning consistently improves knowledge acquisition and retention while maintaining prior knowledge, and it demonstrates favorable training efficiency relative to baselines. The work also outlines avenues for integration with continual learning and broader evaluations, highlighting practical potential for up-to-date knowledge in LLMs.

Abstract

Large language models (LLMs) often struggle to provide up-to-date information due to their one-time training and the constantly evolving nature of the world. To keep LLMs current, existing approaches typically involve continued pre-training on new documents. However, they frequently face difficulties in extracting stored knowledge. Motivated by the remarkable success of the Feynman Technique in efficient human learning, we introduce Self-Tuning, a learning framework aimed at improving an LLM's ability to effectively acquire new knowledge from unseen raw documents through self-teaching. Specifically, we develop a Self-Teaching strategy that augments the documents with a set of knowledge-intensive tasks created in a self-supervised manner, focusing on three crucial aspects: memorization, comprehension, and self-reflection. Additionally, we introduce three Wiki-Newpages-2023-QA datasets to facilitate an in-depth analysis of an LLM's knowledge acquisition ability concerning memorization, extraction, and reasoning. Extensive experimental results on various models, e.g., Llama2-7B reveal that Self-Tuning consistently exhibits superior performance across all knowledge acquisition tasks and excels in preserving previous knowledge.

Figures (8)

• Figure 1: Illustration of the knowledge acquisition task with two standard knowledge injection approaches (in the upper part). Depiction of Self-Tuning for effective knowledge acquisition from unseen raw documents, which significantly enhances factual accuracy compared to the standard approaches (in the lower part).
• Figure 2: Illustration of the proposed Self-Tuning. The framework consists of three stages (in the upper part): $( \textup{\it i})$ Equipping the model with the ability to deeply absorb knowledge from raw documents using the proposed Self-Teaching strategy (in the lower part), along with question-answering capabilities; $( \textup{\it ii})$ Applying the learning strategy acquired in Stage 1 to obtain new knowledge from unseen documents and refining QA skills; $( \textup{\it ii})$ Continuously learning from unseen documents. See Appendix \ref{['sec:example_doc_task']} for the full training document example in Stage 1.
• Figure 3: Training dynamics on Llama2-7Bw.r.t., knowledge memorization, extraction, and retention across different numbers of training epochs. We present the EM scores on NQ datasets to evaluate knowledge retention. The black and red dashed lines represent the baseline closed-book and open-book performances for the knowledge extraction task, respectively.
• Figure 4: Ablation analysis exploring the impact of removing comprehension and self-reflection tasks from the self-teaching tasks for knowledge memorization and acquisition. The proportion of each task type among the self-teaching tasks in the training documents is shown in the upper right corner.
• Figure 5: Distribution histogram of the token count in a document, a question, and an answer for the open-ended generation task from the Wiki-Newpages-2023-10-Bio dataset, respectively.