Learning Dynamic Cognitive Map with Autonomous Navigation

TL;DR

This work addresses navigation in unknown, aliased environments by proposing a dynamic cognitive map that grows over predicted poses within an Active Inference framework. It introduces a principled generative model with variational state inference and structure-learning that expands the cognitive map using predicted, not only observed, states, enabling rapid single-episode learning and robust adaptation to environmental changes. Compared with CSCG, the approach demonstrates faster topology learning, effective remapping after obstacles, and resilient self-localisation, supporting scalable autonomous navigation without prior world dimensions. The methodology has potential impact for real-world robotics, enabling efficient exploration, planning, and re-planning in complex, uncertain environments while managing computational resources through a matrix-based, prediction-driven representation.

Abstract

Inspired by animal navigation strategies, we introduce a novel computational model to navigate and map a space rooted in biologically inspired principles. Animals exhibit extraordinary navigation prowess, harnessing memory, imagination, and strategic decision-making to traverse complex and aliased environments adeptly. Our model aims to replicate these capabilities by incorporating a dynamically expanding cognitive map over predicted poses within an Active Inference framework, enhancing our agent's generative model plasticity to novelty and environmental changes. Through structure learning and active inference navigation, our model demonstrates efficient exploration and exploitation, dynamically expanding its model capacity in response to anticipated novel un-visited locations and updating the map given new evidence contradicting previous beliefs. Comparative analyses in mini-grid environments with the Clone-Structured Cognitive Graph model (CSCG), which shares similar objectives, highlight our model's ability to rapidly learn environmental structures within a single episode, with minimal navigation overlap. Our model achieves this without prior knowledge of observation and world dimensions, underscoring its robustness and efficacy in navigating intricate environments.

Figures (10)

• Figure 1: a) From a full 3 by 3 rooms mini-grid environment gym_minigridours_2024 to b) rooms observation layout as perceived by the agent and the path it has taken between rooms - composed of a line from black to white-, c) shows the agent final internal topological graph (cognitive graph) linking all the locations between them.
• Figure 2: This block diagram outlines the agent's decision-making and mapping process. The agent begins by perceiving an observation, which updates the $A_o$ matrix—representing the observation model—by adjusting its dimensions to incorporate new sensory data. The agent then infers its current state based on this updated model. Next, it predicts the outcomes of possible motions in all directions, enabling it to update the model in relevant directions. The transition models, $B_s$, $B_p$, $A_p$ and $A_o$, are updated with anticipated new transitions (to new state or same state if obstacle), where $B_p$ represents probabilistic transitions between positions due to motion, $B_s$ captures state transition based on imagined spatial structure, and $A_p$ encoding the probability of a position given a state. Using this information, the agent decides on its next action and executes it by moving, repeating the process.
• Figure 3: Factor graph of the POMDP in our generative model, showing transitions from the past to the present (up to time-step $t$) and extending into the future (time-step $t+1$). Past observations are marked in blue, indicating they are known. In the future steps, actions follow a policy $\pi$ influencing the new states and position in orange and new predictions in grey. The position at time $t$, $p_t$, is determined by the policy and the prior position $p_{t-1}$, while the current state $s_t$ is inferred from the observation $o_t$, the position $p_t$, and the previous state $s_{t-1}$. Transitions between states are ruled by the $B$ matrices, which define how prior conditions contribute to the current one considering taken actions. $A$ matrices represent conditional probabilities of the quantities they connect.
• Figure 4: Example of an internal map layout based on initial dimensions, expansion strategy, and a given path in a maze -o) path in orange-. with observed states/rooms observations being the room floor. a) A static map expects 9 rooms/states but lacks connectivity and observation details. A dynamic map can expand indefinitely with b) new observations or c) predictions. a) Requires knowing the environment's size in advance, while b) does not foresee new rooms in un-visited past areas. c) Considers the possibility of every door leading to un-visited rooms.
• Figure 5: From a physical motion to the subsequent cognitive graph update. The first red room is initialised with current observation and predicted motions in the four directions. Going Down and Right holds new unknown states. By going right toward the blue room, the state is updated with a new blue observation and predicted motions in all directions, increasing the confidence in the red-blue room transition and defining it as bi-directional.