Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs Could Autonomously Learn Without External Supervision

Ke Ji, Junying Chen, Anningzhe Gao, Wenya Xie, Xiang Wan, Benyou Wang

TL;DR

Empirical results reveal that Autonomous Learning outstrips the performance of both Pre-training and Supervised Fine-Tuning (SFT), as well as retrieval-augmented methods, and underscores the potential of Autonomous Learning to not only enhance the efficiency and effectiveness of LLM training but also to pave the way for the development of more advanced, self-reliant AI systems.

Abstract

In the quest for super-human performance, Large Language Models (LLMs) have traditionally been tethered to human-annotated datasets and predefined training objectives-a process that is both labor-intensive and inherently limited. This paper presents a transformative approach: Autonomous Learning for LLMs, a self-sufficient learning paradigm that frees models from the constraints of human supervision. This method endows LLMs with the ability to self-educate through direct interaction with text, akin to a human reading and comprehending literature. Our approach eliminates the reliance on annotated data, fostering an Autonomous Learning environment where the model independently identifies and reinforces its knowledge gaps. Empirical results from our comprehensive experiments, which utilized a diverse array of learning materials and were evaluated against standard public quizzes, reveal that Autonomous Learning outstrips the performance of both Pre-training and Supervised Fine-Tuning (SFT), as well as retrieval-augmented methods. These findings underscore the potential of Autonomous Learning to not only enhance the efficiency and effectiveness of LLM training but also to pave the way for the development of more advanced, self-reliant AI systems.

LLMs Could Autonomously Learn Without External Supervision

TL;DR

Empirical results reveal that Autonomous Learning outstrips the performance of both Pre-training and Supervised Fine-Tuning (SFT), as well as retrieval-augmented methods, and underscores the potential of Autonomous Learning to not only enhance the efficiency and effectiveness of LLM training but also to pave the way for the development of more advanced, self-reliant AI systems.

Abstract

In the quest for super-human performance, Large Language Models (LLMs) have traditionally been tethered to human-annotated datasets and predefined training objectives-a process that is both labor-intensive and inherently limited. This paper presents a transformative approach: Autonomous Learning for LLMs, a self-sufficient learning paradigm that frees models from the constraints of human supervision. This method endows LLMs with the ability to self-educate through direct interaction with text, akin to a human reading and comprehending literature. Our approach eliminates the reliance on annotated data, fostering an Autonomous Learning environment where the model independently identifies and reinforces its knowledge gaps. Empirical results from our comprehensive experiments, which utilized a diverse array of learning materials and were evaluated against standard public quizzes, reveal that Autonomous Learning outstrips the performance of both Pre-training and Supervised Fine-Tuning (SFT), as well as retrieval-augmented methods. These findings underscore the potential of Autonomous Learning to not only enhance the efficiency and effectiveness of LLM training but also to pave the way for the development of more advanced, self-reliant AI systems.
Paper Structure (33 sections, 11 equations, 8 figures, 8 tables, 1 algorithm)

This paper contains 33 sections, 11 equations, 8 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Conceptualization: Autonomous Learning
  3. Problem Statement
  4. Existing Learning for LLMs
  5. Pre-training
  6. Supervised Fine-Tuning (SFT)
  7. Autonomous Learning
  8. Autonomous Learning (AL)
  9. Methodology: Autonomous Learning
  10. Stage 1. Open-Book Learning
  11. Stage 2. Closed-book Learning
  12. The Benefits of Autonomous Learning
  13. Experiments
  14. Target Domain With Various Scales and Downstream Tasks
  15. Experiments Setup
  16. ...and 18 more sections

Figures (8)

  • Figure 1: An ideal learning system should learn autonomously to determine what to learn, how to learn and why to learn.
  • Figure 2: The prompt for document comprehension. [domain-specific document] indicates the document $d$ to be learned.
  • Figure 3: The performance gap between open-book learning and closed-book learning. Epoch 0 stands for the performance of initial model. Epochs 4 to 6 represent the 1st, 2nd, and 3rd epochs of closed-book learning, respectively.
  • Figure 4: The trend of logits variation for open-book answers and closed-book answers on the MedQA-cn dataset.
  • Figure 5: Reward performance on MedQA-cn dataset.
  • ...and 3 more figures