Spatial and Temporal Hierarchy for Autonomous Navigation using Active Inference in Minigrid Environment

TL;DR

This paper presents a scalable hierarchical active inference model for autonomous navigation, exploration, and goal-oriented behaviour that uses visual observation and motion perception to combine curiosity-driven exploration with goal-oriented behaviour.

Abstract

Robust evidence suggests that humans explore their environment using a combination of topological landmarks and coarse-grained path integration. This approach relies on identifiable environmental features (topological landmarks) in tandem with estimations of distance and direction (coarse-grained path integration) to construct cognitive maps of the surroundings. This cognitive map is believed to exhibit a hierarchical structure, allowing efficient planning when solving complex navigation tasks. Inspired by human behaviour, this paper presents a scalable hierarchical active inference model for autonomous navigation, exploration, and goal-oriented behaviour. The model uses visual observation and motion perception to combine curiosity-driven exploration with goal-oriented behaviour. Motion is planned using different levels of reasoning, i.e., from context to place to motion. This allows for efficient navigation in new spaces and rapid progress toward a target. By incorporating these human navigational strategies and their hierarchical representation of the environment, this model proposes a new solution for autonomous navigation and exploration. The approach is validated through simulations in a mini-grid environment.

Figures (17)

• Figure 1: Our generative model unrolled in time and levels as defined in Eq.\ref{['eq:HAIF']}. The left figure shows the graphical model of the 3-layer hierarchical Active inference model consisting of a) the cognitive map, b) the allocentric model, and c) the egocentric model, each operating at a different time scale. The orange circles represent latent states that have to be inferred, the blue circles denote observable outcomes and the white circles are internal variables to be inferred. The right part visualises the representation at each layer. The cognitive map is represented as d) a topological graph composed of all the locations ($l$) and their connections, in which each location is stored in a distinct node. The allocentric model e) infers place representations ($z$) by integrating sequences of state ($s$) and poses ($p$), from which the room structure can be generated. The egocentric model f) imagines future observations given the current position, state ($s$), and possible actions ($a$). Here o) depicts an actual observation ($o$) and the predicted observations of the possible actions turn left i), move forward ii), and turn right iii).
• Figure 2: Generative model for the egocentric level: POMDP depicting the model transition from past and present (up to timestep $\tau$) to future (from timestep $\tau+1$). A state $s_\tau$ is determined by the corresponding observation $o_\tau$ and influenced by the previous state $s_{\tau-1}$ and action $a_{\tau-1}$, generating the supplementary collision observation $c_\tau$. The action as well as both observations are assumed observable, indicated by the blue colour. In the future, the actions are defined by a policy $\pi$ influencing the new states in orange and new predictions in grey.
• Figure 3: Generative model for the allocentric level as a Bayesian network. One place is considered and described by a latent variable $z$. The observations $o_t$ depend on both the place described by $z$ and the agent’s position $p_t$. From 0 to t, the positions have been visited and are used to infer a belief over the joint distribution. Future viewpoint $p_{t+1}$ has not been visited or observed yet. Observed variables are shown in blue, while inferred variables are shown in white, and predictions are presented in grey.
• Figure 4: Illustration depicting L-shaped paths encompassing the upper right quadrant of an area surrounding the agent. The chosen look-ahead distance in this scenario is 2.
• Figure 5: Evolution of the place representation in a room as new observations are provided by the moving agent (red triangle). The model is able to correctly reconstruct the structure of the room as observations are collected.