Knowledge-Centric Metacognitive Learning

TL;DR

This work introduces knowledge-centric metacognitive learning based on three key principles: natural abstractions, knowledge-guided interactions through interpretation, and the composition of interactions for problem solving.

Abstract

Interactions are central to intelligent reasoning and learning abilities, with the interpretation of abstract knowledge guiding meaningful interaction with objects in the environment. While humans readily adapt to novel situations by leveraging abstract knowledge acquired over time, artificial intelligence systems lack principled mechanisms for incorporating abstract knowledge into learning, leading to fundamental challenges in the emergence of intelligent and adaptive behavior. To address this gap, we introduce knowledge-centric metacognitive learning based on three key principles: natural abstractions, knowledge-guided interactions through interpretation, and the composition of interactions for problem solving. Knowledge learning facilitates the acquisition of abstract knowledge and the association of interactions with knowledge, while object interactions guided by abstract knowledge enable the learning of transferable interaction concepts, abstract reasoning, and generalization. This metacognitive mechanism provides a principled approach for integrating knowledge into reinforcement learning and offers a promising pathway toward intelligent and adaptive behavior in artificial intelligence, robotics, and autonomous systems.

Figures (6)

• Figure 1: a) Illustration of metacognitive approach in a navigation domain, where an agent must develop an abstract understanding of its environment and interpret it to achieve the goal of retrieving a blue ball located behind a locked door, blocked by obstacles, with the key hidden inside a box. b) A knowledge centric metacognitive learning approach exploits inherent object abstractions and relations to generate interaction goals and compose interactions, thereby enabling the emergence of intelligent and flexible behavior.
• Figure 2: The metacognitive learning framework comprises three levels: knowledge or meta ($\mathcal{K}$), interaction ($\mathcal{I}$), and execution ($\mathcal{X}$), based on abstraction and computation. The knowledge level contains type space knowledge graphs that represent higher-order abstractions, while the instance space comprises object instance graphs. The interaction level serves as an intermediary, making recommendations for interactions that the agent executes at the execution level. Computation proceeds through cycles of recognition, recommendation, execution, and interaction, to collect experiences for updates to both the meta-level and interaction policies.
• Figure 3: Environments: ObstructedMaze-Full is used to experimentally evaluate the agent's knowledge learning and flexible task solving capabilities, while ObstructedMaze-1Dlhb is used to evaluate the transferability of interaction policies.
• Figure 4: Agents learning to interact : The learning progress of both the agents KIX-A (blue) and KIX-R (red) is shown as returns over steps, demonstrating the learning of meta-level as well as interaction policies such as 'reach' (only applicable for KIX-R), 'open', 'open with key', 'reveal', 'pickup', 'drop' (not shown) in the ObstructedMaze-Full environment.
• Figure 5: Evaluation return profiles of agents performing tasks show that the agents benefit from meta-level guided interactions.