Learning from Hallucinating Critical Points for Navigation in Dynamic Environments

TL;DR

The paper addresses the challenge of learning to navigate in dynamic environments by generating rich dynamic obstacle datasets without costly demonstrations. It introduces Learning from Hallucinating Critical Points (LfH-CP), a two-stage, self-supervised framework that first identifies critical obstacle configurations and then samples diverse trajectories through these points, guided by a novel dataset-diversity metric. The approach reframes planning under the Learning from Hallucination paradigm, renders obstacle trajectories consistent with optimal plans, and trains a transformer-based motion planner from the synthetic data. Empirical results in simulation show that LfH-CP yields substantially greater dataset coverage than prior hallucination methods and improves navigation success rates compared to baselines, highlighting its potential for scalable, safe learning in dynamic domains.

Abstract

Generating large and diverse obstacle datasets to learn motion planning in environments with dynamic obstacles is challenging due to the vast space of possible obstacle trajectories. Inspired by hallucination-based data synthesis approaches, we propose Learning from Hallucinating Critical Points (LfH-CP), a self-supervised framework for creating rich dynamic obstacle datasets based on existing optimal motion plans without requiring expensive expert demonstrations or trial-and-error exploration. LfH-CP factorizes hallucination into two stages: first identifying when and where obstacles must appear in order to result in an optimal motion plan, i.e., the critical points, and then procedurally generating diverse trajectories that pass through these points while avoiding collisions. This factorization avoids generative failures such as mode collapse and ensures coverage of diverse dynamic behaviors. We further introduce a diversity metric to quantify dataset richness and show that LfH-CP produces substantially more varied training data than existing baselines. Experiments in simulation demonstrate that planners trained on LfH-CP datasets achieves higher success rates compared to a prior hallucination method.

Figures (7)

• Figure 1: A mobile robot's optimal path around the hallucinated critical point and its generated paths using LfH-CP.
• Figure 2: The LfH-CP method visualized for motion plan (green) and one obstacle (red). We first learn where the obstacle should be to make our plan optimal using Eqn. \ref{['eqn::psi_phase1']} (Fig. \ref{['fig:a']}). Then using Eqn. \ref{['eqn::psi_phase2']}, we learn where the obstacle should be temporally to make the plan optimal (Fig. \ref{['fig:b']},\ref{['fig:c']}). Finally using Eqn. \ref{['eqn::generation_function']}, numerous obstacle trajectories can be generated from the obstacle's critical point (Fig. \ref{['fig:d']}).
• Figure 3: Obstacle critical points map.
• Figure 4: 100 obstacle trajectories are generated from each critical point. The robot trajectory (green) remains collision-free and near-optimal, regardless of the variations and combinations of obstacle trajectories. That is, the same robot trajectory is optimal and collision-free for any single combination of these $100^N$ possible obstacle trajectory sets. Fig. \ref{['fig:A']} show a total of 200 obstacle trajectories generated from $N=2$ critical points, while Fig. \ref{['fig:B']} show a total of 300 obstacle trajectories generated from $N=3$ critical points.
• Figure 5: Reconstructed trajectories from Phase 1 (P1) and Phase 2 (P2) using obstacles at their critical points. Trajectories remain visually close to each other, with only minor variations in reconstruction loss.