GRL-SNAM: Geometric Reinforcement Learning with Path Differential Hamiltonians for Simultaneous Navigation and Mapping in Unknown Environments
Aditya Sai Ellendula, Yi Wang, Minh Nguyen, Chandrajit Bajaj
TL;DR
GRL-SNAM addresses the Simultaneous Navigation and Mapping (SNAM) problem by introducing a geometric reinforcement learning framework that learns to shape a reduced Hamiltonian over a shared energy landscape using only local observations. It decomposes navigation, sensing, and deformation into three modular policies whose interactions are governed by energy terms that are adapted online via QoIs, enabling stable forward-rollouts without constructing global maps. The method combines offline learning of energy components with online, stagewise adaptation, achieving high navigation quality with minimal mapping effort in deformable-ring and dungeon navigation tasks, and outperforming both classical planners and standard deep RL baselines under identical sensing constraints. The key findings show interpretable energy-field behavior, robust adaptation to perturbations, and strong transfer to unseen layouts, suggesting that physics-informed Hamiltonian supervision offers a practical, data-efficient alternative to reward-centric policy learning for SNAM. Overall, GRL-SNAM demonstrates that embedding geometric priors and online energy reshaping into RL yields efficient, safe, and scalable navigation in unknown environments.
Abstract
We present GRL-SNAM, a geometric reinforcement learning framework for Simultaneous Navigation and Mapping(SNAM) in unknown environments. A SNAM problem is challenging as it needs to design hierarchical or joint policies of multiple agents that control the movement of a real-life robot towards the goal in mapless environment, i.e. an environment where the map of the environment is not available apriori, and needs to be acquired through sensors. The sensors are invoked from the path learner, i.e. navigator, through active query responses to sensory agents, and along the motion path. GRL-SNAM differs from preemptive navigation algorithms and other reinforcement learning methods by relying exclusively on local sensory observations without constructing a global map. Our approach formulates path navigation and mapping as a dynamic shortest path search and discovery process using controlled Hamiltonian optimization: sensory inputs are translated into local energy landscapes that encode reachability, obstacle barriers, and deformation constraints, while policies for sensing, planning, and reconfiguration evolve stagewise via updating Hamiltonians. A reduced Hamiltonian serves as an adaptive score function, updating kinetic/potential terms, embedding barrier constraints, and continuously refining trajectories as new local information arrives. We evaluate GRL-SNAM on two different 2D navigation tasks. Comparing against local reactive baselines and global policy learning references under identical stagewise sensing constraints, it preserves clearance, generalizes to unseen layouts, and demonstrates that Geometric RL learning via updating Hamiltonians enables high-quality navigation through minimal exploration via local energy refinement rather than extensive global mapping. The code is publicly available on \href{https://github.com/CVC-Lab/GRL-SNAM}{Github}.