Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Knowledge Boundary of Large Language Models: A Survey

Moxin Li, Yong Zhao, Wenxuan Zhang, Shuaiyi Li, Wenya Xie, See-Kiong Ng, Tat-Seng Chua, Yang Deng

TL;DR

This survey formalizes the knowledge boundary of LLMs into three boundaries and a four-type taxonomy, addressing limitations of prior work. It reviews undesired behaviors, identification methods, and mitigation strategies through a unified framework. It also discusses benchmarks, knowledge mechanisms, and challenges for long-form factuality and generalization. The work aims to guide future research toward trustworthy knowledge use and boundary-aware model upgrades.

Abstract

Although large language models (LLMs) store vast amount of knowledge in their parameters, they still have limitations in the memorization and utilization of certain knowledge, leading to undesired behaviors such as generating untruthful and inaccurate responses. This highlights the critical need to understand the knowledge boundary of LLMs, a concept that remains inadequately defined in existing research. In this survey, we propose a comprehensive definition of the LLM knowledge boundary and introduce a formalized taxonomy categorizing knowledge into four distinct types. Using this foundation, we systematically review the field through three key lenses: the motivation for studying LLM knowledge boundaries, methods for identifying these boundaries, and strategies for mitigating the challenges they present. Finally, we discuss open challenges and potential research directions in this area. We aim for this survey to offer the community a comprehensive overview, facilitate access to key issues, and inspire further advancements in LLM knowledge research.

Knowledge Boundary of Large Language Models: A Survey

TL;DR

This survey formalizes the knowledge boundary of LLMs into three boundaries and a four-type taxonomy, addressing limitations of prior work. It reviews undesired behaviors, identification methods, and mitigation strategies through a unified framework. It also discusses benchmarks, knowledge mechanisms, and challenges for long-form factuality and generalization. The work aims to guide future research toward trustworthy knowledge use and boundary-aware model upgrades.

Abstract

Although large language models (LLMs) store vast amount of knowledge in their parameters, they still have limitations in the memorization and utilization of certain knowledge, leading to undesired behaviors such as generating untruthful and inaccurate responses. This highlights the critical need to understand the knowledge boundary of LLMs, a concept that remains inadequately defined in existing research. In this survey, we propose a comprehensive definition of the LLM knowledge boundary and introduce a formalized taxonomy categorizing knowledge into four distinct types. Using this foundation, we systematically review the field through three key lenses: the motivation for studying LLM knowledge boundaries, methods for identifying these boundaries, and strategies for mitigating the challenges they present. Finally, we discuss open challenges and potential research directions in this area. We aim for this survey to offer the community a comprehensive overview, facilitate access to key issues, and inspire further advancements in LLM knowledge research.

Paper Structure

This paper contains 71 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Definition of Knowledge Boundary
  3. Undesired Behaviours
  4. Untruthful Responses Misled by Context
  5. Untruthful Context
  6. Irrelevant Context
  7. Factuality Hallucinations
  8. Deficiency of Domain-specific Knowledge
  9. Outdated Knowledge
  10. Overconfidence on Unknown Knowledge
  11. Truthful but Undesired Responses
  12. Random Responses to Ambiguous Knowledge
  13. Biased Responses to Controversial Knowledge
  14. Identification of Knowledge Boundary
  15. Uncertainty Estimation
  16. ...and 56 more sections

Figures (4)

  • Figure 1: Illustration of the knowledge boundaries and knowledge taxonomy of LLM. The dashed circle in (a) represents the "truly" prompt-agnostic known knowledge $k$, which can be verified by any expression in $Q_k$. In practice, however, the prompt-agnostic nature of $k$ can only be approximated using a limited subset $\hat{Q}_k\subseteq Q_k$. As a result, the outward knowledge boundary is depicted with an irregularly shaped line to reflect this approximation.
  • Figure 2: Summary of the mitigation techniques for prompt-sensitive known knowledge.
  • Figure 3: Summary of the mitigation techniques for model-specific unknown knowledge.
  • Figure 4: The main content flow and categorization of this survey.