Probing the Knowledge Boundary: An Interactive Agentic Framework for Deep Knowledge Extraction

TL;DR

The paper tackles the problem of characterizing what deep learning language models actually know by defining the knowledge boundary as a verifiable set of knowledge atoms within a topic. It introduces an Interactive Agentic Framework that combines four adaptive exploration policies with a rigorous three-stage Knowledge Processor to saturate and validate extracted knowledge. Through formal problem formulation, a Pareto-frontier analysis, and four controlled experiments, it reveals a knowledge scaling law where larger models expose more knowledge, identifies a Pass@1 vs Pass@k trade-off with specialization, and shows that training data composition shapes model-specific knowledge profiles. The framework offers a practical auditing method for evaluating model knowledge before deployment and provides design insights for pretraining and fine-tuning strategies. Overall, it demonstrates how hierarchical, saturation-aware exploration paired with semantic deduplication and domain auditing can illuminate the latent knowledge structure of black-box LLMs.

Abstract

Large Language Models (LLMs) can be seen as compressed knowledge bases, but it remains unclear what knowledge they truly contain and how far their knowledge boundaries extend. Existing benchmarks are mostly static and provide limited support for systematic knowledge probing. In this paper, we propose an interactive agentic framework to systematically extract and quantify the knowledge of LLMs. Our method includes four adaptive exploration policies to probe knowledge at different granularities. To ensure the quality of extracted knowledge, we introduce a three-stage knowledge processing pipeline that combines vector-based filtering to remove exact duplicates, LLM-based adjudication to resolve ambiguous semantic overlaps, and domain-relevance auditing to retain valid knowledge units. Through extensive experiments, we find that recursive taxonomy is the most effective exploration strategy. We also observe a clear knowledge scaling law, where larger models consistently extract more knowledge. In addition, we identify a Pass@1-versus-Pass@k trade-off: domain-specialized models achieve higher initial accuracy but degrade rapidly, while general-purpose models maintain stable performance during extended extraction. Finally, our results show that differences in training data composition lead to distinct and measurable knowledge profiles across model families.

Figures (4)

• Figure 1: Overview of the Interactive Agentic Framework. (a) Agentic Exploration Policies actively probe the black-box LLM; the diagram illustrates the Recursive Taxonomy strategy. (b) Knowledge Processor transforms raw outputs into valid points via three-stage filtering: Vector Filtering, LLM Adjudication, and Domain Auditing.
• Figure 2: Aggregated Pareto Frontier. Mean yield ratio vs. cumulative token cost for 8 exploration strategies on Llama-3.1-405B, averaged across three domains. Shaded regions indicate cross-domain standard deviation. Yields are normalized to Taxonomy (L2W2) at saturation.
• Figure 3: Specialized vs General Models. Accuracy (top) and recall (bottom) curves for Qwen 2.5-7B Instruct vs. Coder across three domains. Deep Learning and ML Systems are target domains for the Coder model; Probabilistic Methods is a non-target domain.
• Figure 4: Cross-Series Comparison. Accuracy (top) and recall (bottom) curves for three $\sim$7B models from different organizations: Llama-3.1-8B, Qwen-2.5-7B, and DeepSeek-R1-Distill-7B.