Exploring and Learning Structure: Active Inference Approach in Navigational Agents

TL;DR

This work tackles rapid structure learning for navigation in aliased environments by integrating a dynamically expanding topological cognitive map with an Active Inference framework within a hierarchical POMDP. It learns model parameters via minimising the free energy $F$ while planning with expected free energy $G$, allowing the internal map to grow as new predicted beliefs arise. The approach emphasizes imagination-driven exploration and information gain to drive map expansion, achieving faster topology learning than Clone-Structured Graphs in one-shot scenarios. The results suggest strong potential for robust, biologically inspired navigation in unknown or changing environments and point to real-world robotics and street-level navigation applications like StreetLearn.

Abstract

Drawing inspiration from animal navigation strategies, we introduce a novel computational model for navigation and mapping, rooted in biologically inspired principles. Animals exhibit remarkable navigation abilities by efficiently using memory, imagination, and strategic decision-making to navigate complex and aliased environments. Building on these insights, we integrate traditional cognitive mapping approaches with an Active Inference Framework (AIF) to learn an environment structure in a few steps. Through the incorporation of topological mapping for long-term memory and AIF for navigation planning and structure learning, our model can dynamically apprehend environmental structures and expand its internal map with predicted beliefs during exploration. Comparative experiments with the Clone-Structured Graph (CSCG) model highlight our model's ability to rapidly learn environmental structures in a single episode, with minimal navigation overlap. this is achieved without prior knowledge of the dimensions of the environment or the type of observations, showcasing its robustness and effectiveness in navigating ambiguous environments.

Figures (5)

• Figure 1: From the mini-grid environment gym_minigridours_2024 with different rooms annotated by colour to the path our agent took -from black to white- to form a successful exploration correctly linking all the rooms up to the agent's internal topological graph with the state associated to each room.
• Figure 2: factor graph POMDP of our generative model transitioning from past and present (up to time-step $t$) to future (time-step $t+1$). The pose $p_t$ is inferred from the previous pose $p_{t-1}$ and the action from policy $\pi$, while the state $s_t$ determined by the corresponding observation $o_t$ and influenced by the previous state $s_{t-1}$, pose $p_{t}$ and action $a_{t-1}$. Past actions and observations are assumed observable, indicated by a blue colour. In the future, the actions are defined by a policy $\pi$ influencing the new states and position in orange and new predictions in grey.
• Figure 3: The average steps are depicted on a logarithmic scale. Remarkably, our agent achieves all tasks in significantly fewer steps compared to the CSCG model. The oracle sets the benchmark, representing the minimum steps necessary to visit all rooms once. Additionally, an aliased room signifies the recurrence of identical observations across various locations, posing a challenge as it could mislead the agent regarding its current position.
• Figure 4: Example of a successful transition representation between positions in a 3x3 grid map. Each state in the plot is paired with its corresponding ground-truth pose for clarity (pose, state). The intensity of colour in the figure indicates the level of certainty the agent has about the transition.
• Figure 5: Exploration of a T-maze starting at the base of the T. a) depicts the full path as a line transitioning from black to white. b) showcases, from top to bottom columns one to two, the agent -represented as an X- imagined optimal policies. Darker colours indicate higher expected free energy.