Agentic Knowledgeable Self-awareness

TL;DR

This paper tackles the brittleness and inefficiency of existing agent planning by introducing agentic knowledgeable self-awareness, a data-centric paradigm that enables LLM-based agents to autonomously regulate knowledge usage according to situational demands. The KnowSelf framework constructs a lightweight knowledge base and a situation-aware data pipeline that marks trajectories with tokens signaling fast thinking, slow thinking, or knowledge-based thinking, and trains agents in two stages (SFT then a policy-aware objective with a DPO/RPO blend). Empirical results on ALFWorld and WebShop show KnowSelf achieves superior planning performance with minimal external knowledge, and scaling analyses reveal strong generalization and a late-layer emergence of self-awareness. The approach reduces knowledge injection costs, improves robustness to distribution shifts, and offers a concrete path toward more autonomous, resource-efficient knowledge-aware agents with practical implications for real-world planning tasks.

Abstract

Large Language Models (LLMs) have achieved considerable performance across various agentic planning tasks. However, traditional agent planning approaches adopt a "flood irrigation" methodology that indiscriminately injects gold trajectories, external feedback, and domain knowledge into agent models. This practice overlooks the fundamental human cognitive principle of situational self-awareness during decision-making-the ability to dynamically assess situational demands and strategically employ resources during decision-making. We propose agentic knowledgeable self-awareness to address this gap, a novel paradigm enabling LLM-based agents to autonomously regulate knowledge utilization. Specifically, we propose KnowSelf, a data-centric approach that applies agents with knowledgeable self-awareness like humans. Concretely, we devise a heuristic situation judgement criterion to mark special tokens on the agent's self-explored trajectories for collecting training data. Through a two-stage training process, the agent model can switch between different situations by generating specific special tokens, achieving optimal planning effects with minimal costs. Our experiments demonstrate that KnowSelf can outperform various strong baselines on different tasks and models with minimal use of external knowledge. Code is available at https://github.com/zjunlp/KnowSelf.

Figures (5)

• Figure 1: Agentic Knowledgeable Self-awareness.
• Figure 2: The framework of our KnowSelf. Firstly, we mark self-explored trajectories with special tokens according to the situation judgement criterion to form the training data. Secondly, we apply a two-stage training framework to teach the agent model knowledgeable self-awareness abilities. Finally, the agent model identifies different situations by generating specific special tokens during inference.
• Figure 3: (a) Ablation studies for KnowSelf on ALFWorld. (b) Generalization ability of KnowSelf. We select three simple task types in ALFWorld as training sets and the other three kinds of tasks as test sets. (c) Scaling law of agentic knowledgeable self-awareness. We analyze aspects of the model and data scales on ALFWorld.
• Figure 4: Mechainsm of agentic knowledgeable self-awareness. We calculate the average probabilities of tokens representing various situations at each layer of the Transformer across both knowledgeable thinking (w/ Know) and fast thinking (w/o Know) scenarios. A more detailed experiment setup can be seen in Appendix \ref{['app:mechanism']}.
• Figure 5: Case Study. SOTA LLMs fall short in agentic knowledgeable self-awareness through only prompting.