Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Into the Unknown: Self-Learning Large Language Models

Teddy Ferdinan, Jan Kocoń, Przemysław Kazienko

TL;DR

A self-learning LLM framework is proposed that enables an LLM to independently learn previously unknown knowledge through self-assessment of their own hallucinations, and a concept called Point in the Unknown (PiU) to identify atomic knowledge unknown to a model, along with four methods for automatic PiUs identification.

Abstract

We address the main problem of self-learning LLM: the question of what to learn. We propose a self-learning LLM framework that enables an LLM to independently learn previously unknown knowledge through self-assessment of their own hallucinations. We introduce a concept called Point in the Unknown (PiU) to identify atomic knowledge unknown to a model, along with four methods for automatic PiUs identification, facilitating the creation of a self-learning loop that focuses exclusively on the absorption of currently unknown knowledge into the model. Additionally, we developed evaluation metrics to gauge an LLM's self-learning capability. Our experiments revealed that LLMs with at least 3B parameters that have undergone some instruction training would be able to perform self-learning well. We further proved the effectiveness of self-learning by comparing the performance of a model that has undergone self-learning to a model that has not. Our self-learning concept allows more efficient LLM updates and opens new perspectives for LLM knowledge exchange.

Into the Unknown: Self-Learning Large Language Models

TL;DR

A self-learning LLM framework is proposed that enables an LLM to independently learn previously unknown knowledge through self-assessment of their own hallucinations, and a concept called Point in the Unknown (PiU) to identify atomic knowledge unknown to a model, along with four methods for automatic PiUs identification.

Abstract

We address the main problem of self-learning LLM: the question of what to learn. We propose a self-learning LLM framework that enables an LLM to independently learn previously unknown knowledge through self-assessment of their own hallucinations. We introduce a concept called Point in the Unknown (PiU) to identify atomic knowledge unknown to a model, along with four methods for automatic PiUs identification, facilitating the creation of a self-learning loop that focuses exclusively on the absorption of currently unknown knowledge into the model. Additionally, we developed evaluation metrics to gauge an LLM's self-learning capability. Our experiments revealed that LLMs with at least 3B parameters that have undergone some instruction training would be able to perform self-learning well. We further proved the effectiveness of self-learning by comparing the performance of a model that has undergone self-learning to a model that has not. Our self-learning concept allows more efficient LLM updates and opens new perspectives for LLM knowledge exchange.
Paper Structure (33 sections, 7 equations, 2 figures, 4 tables)

This paper contains 33 sections, 7 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Why Self-Learning is Needed
  4. The concept of Point in the Unknown (PiU)
  5. Methods for Identification of PiUs
  6. External Prompt (extrinsic)
  7. Open Generation (intrinsic)
  8. Induced Generation (intrinsic)
  9. Oracle-Selected (intrinsic)
  10. Self-Learning LLM
  11. Self-Questioning
  12. Knowledge Searching
  13. Model Training
  14. Metrics for choosing a base model for self-learning
  15. Curiosity Score
  16. ...and 18 more sections

Figures (2)

  • Figure 1: The illustrative space of knowledge embeddings reduced to two dimensions. It visualizes our four methods for the identification of Points in the Unknown (PiUs), later exploited in the self-learning loop. Dashed lines are the borders of the Known regions (darker green) -- hallucination score thresholds. Out of them are the Unknown regions (lighter green). White points indicate prompts related to knowledge already known to the model, while red points indicate PiUs. Different shapes depict different methods: (1) circles represent extrinsic (external) triggers, i.e., user queries or trending topics; (2) crosses denote open questions-prompts generated by the model itself within a given topic represented by a dotted line; (3) triangles represent the induced questions generated within a topic using 5W+1H; (4) stars indicate the random sampling by selecting random points in the embedding space.
  • Figure 2: Illustration of self-learning LLM with intrinsic inspiration.