A Reinforcement Learning-Based Model for Mapping and Goal-Directed Navigation Using Multiscale Place Fields

TL;DR

This work presents a hippocampal-inspired reinforcement learning model for navigation that deploys parallel multiscale place-field layers, a replay-based reward mechanism, and a dynamic scale fusion strategy. By combining coarse and fine spatial representations with scale-aware value propagation, the approach achieves faster learning and more efficient paths than single-scale baselines, demonstrated through pretrained-map and online-learning experiments across varied environments. Key contributions include the multiscale PC stack, scale-specific reward maps, and a variation-based fusion method that adaptively weights scales by directional reward structure. The findings suggest multiscale spatial representations can robustly unify global guidance with local precision, offering practical benefits for real-world robotic navigation and planning under partial observability.

Abstract

Autonomous navigation in complex and partially observable environments remains a central challenge in robotics. Several bio-inspired models of mapping and navigation based on place cells in the mammalian hippocampus have been proposed. This paper introduces a new robust model that employs parallel layers of place fields at multiple spatial scales, a replay-based reward mechanism, and dynamic scale fusion. Simulations show that the model improves path efficiency and accelerates learning compared to single-scale baselines, highlighting the value of multiscale spatial representations for adaptive robot navigation.

Figures (8)

• Figure 1: Multiscale system architecture integrating sensory inputs, neural processing layers, and decision-making components. The Head Direction Network processes global heading data, which modulates the activations of BVC and PC layers across multiple scales. The PC layer activations at each scale are then passed to their respective Reward Cell Layer, which returns a potential reward value for each possible heading. These reward values are then aggregated by a fusion module, after which a single, optimal action is chosen and executed.
• Figure 2: Activation patterns of three representative place cells across spatial scales ($\sigma_r = 0.5, 2.0,$ and $4.0$ m) in a $20 \times 20$ m environment. Broader $\sigma_r$ values produce larger receptive fields, indicating coarser spatial representations. Colors denote activation strength.
• Figure 3: Reward-cell activation patterns for each of the three spatial scales, corresponding to a reward located in the lower-left corner of a $20 \times 20$ m environment. Colors indicate cell activation strength
• Figure 4: Reward fusion architecture. Preplay generates predicted place-cell activity for each heading and scale; directional variation in the reward maps determines the weighting factors $\alpha_k$, and the weighted sum identifies the best action $a^*$.
• Figure 5: Randomly-selected trajectories. In multiscale runs, the color along each path segment indicates the scale that dominates decision-making, though scales that do not appear as dominant may still be active.