Embodied World Models Emerge from Navigational Task in Open-Ended Environments

TL;DR

The paper investigates whether embodied sensorimotor interaction suffices for the spontaneous emergence of compact world models in artificial agents. By training a gated-recurrent agent in thousands of open-ended 10×10 mazes with sparse rewards and partial observation, the authors cast the closed agent–environment loop as a Hybrid Dynamical System and demonstrate stable limit-cycle strategies. They introduce Ridge Representation to map entire trajectories into fixed-size behavioral images and use Canonical Correlation Analysis to reveal a high-dimensional linear alignment between neural activations and Ridge-based behavioral geometry, with causal interventions confirming the importance of highly correlated neural dimensions. Collectively, the work provides mechanistic evidence that embodied interaction can produce interpretable, transferable spatial representations, and offers a principled toolkit (HDS, Ridge, CCA, and cyclic stimulation) for diagnosing embodied intelligence in navigation agents.

Abstract

Spatial reasoning in partially observable environments has often been approached through passive predictive models, yet theories of embodied cognition suggest that genuinely useful representations arise only when perception is tightly coupled to action. Here we ask whether a recurrent agent, trained solely by sparse rewards to solve procedurally generated planar mazes, can autonomously internalize metric concepts such as direction, distance and obstacle layout. After training, the agent consistently produces near-optimal paths in unseen mazes, behavior that hints at an underlying spatial model. To probe this possibility, we cast the closed agent-environment loop as a hybrid dynamical system, identify stable limit cycles in its state space, and characterize behavior with a Ridge Representation that embeds whole trajectories into a common metric space. Canonical correlation analysis exposes a robust linear alignment between neural and behavioral manifolds, while targeted perturbations of the most informative neural dimensions sharply degrade navigation performance. Taken together, these dynamical, representational, and causal signatures show that sustained sensorimotor interaction is sufficient for the spontaneous emergence of compact, embodied world models, providing a principled path toward interpretable and transferable navigation policies.

Figures (23)

• Figure 1: Conceptual illustration contrasting comprehension through observation (World Model) with comprehension through interaction (Embodied Cognition). On the left, the elevator with upward/downward indicators and a figure facing away symbolizes the “observation-based” paradigm, where an agent relies primarily on external cues to construct a world model. On the right, the figure on a spiraling staircase underscores the importance of physical actions and sensorimotor feedback—hallmarks of embodied cognition. The bidirectional arrow in the center highlights the transition between these two frameworks, suggesting how direct interaction can yield more deeply grounded internal representations in complex environments.
• Figure 2: Illustration of efficient navigation in random mazes over time. Each panel displays a distinct 10×10 maze with black squares indicating obstacles and white squares denoting traversable cells. The agent’s path (green line) begins at the green circle (start) and evolves across consecutive attempts, progressively shortening and refining as the agent internalizes key layout information. By preserving its recurrent hidden state after each goal reach, the agent rapidly transitions from exploratory, sometimes circuitous routes to near-optimal navigation strategies. The horizontal axis depicts the temporal sequence of attempts, highlighting how path length and detours decrease with continued interactions.
• Figure 3: Rationale for hybrid dynamical sampling beyond direct environment trajectories. (a) Illustration of raw maze trajectories (Q) alongside corresponding neural activations (X) for two different maze layouts. The maze panels (left) show agent movement (green circles and lines) in grids with obstacles (black squares); at each step, a locally observed state triggers updates in the network’s hidden units, depicted here as vertical bars (red/yellow) reflecting activation magnitudes. Directly recording such environment trajectories can introduce biases from suboptimal or exploratory paths, making it difficult to isolate the agent’s intended strategy. (b) Conceptual diagram of the joint (Q×X) space, in which both physical coordinates (d4,d5) and neural dimensions (d1,d2,d3) jointly evolve in a closed loop. Examining this higher-dimensional hybrid system enables cleaner identification of stable or near-optimal strategy attractors, overcoming limitations of naive trajectory sampling.
• Figure 4: Initial observations of limit-cycle sampling in the hybrid Q×X space. (a) Schematic of a partially obstructed 10×10 maze, illustrating sequential partial resets. The agent progresses from Q1 to Q7 (green cells) while retaining its hidden state between attempts, thereby refining its path. (b) Sample 3D projection (e.g., via PCA) of hidden-state vectors recorded during these traversals. Clusters or recurrent loops in the activation space (one highlighted by the red ellipse) suggest an emerging limit cycle as the agent stabilizes its route. (c) Maze illustration showing how the agent—upon reaching Q7—is reset back to Q1 while preserving its internal states. This cyclical “Q1→Q7→Q1” process creates a closed-loop trajectory in physical space, reinforcing a stable limit cycle both in the environment and the agent’s neural representation.
• Figure 5: PCA projections revealing ring-like structures in the agent’s hidden states. (a) A 3D scatter plot of HDS-sampled neural activations, showing a pronounced circular/elliptical distribution. Each point corresponds to a single time step in the agent’s navigation, with colors denoting different segments or trials. This “ring” indicates that the network’s internal representation systematically organizes key variables (e.g., direction or distance) around a low-dimensional cycle. (b) The same plot with a directional cue (red arrow), underscoring how the agent’s hidden states transition smoothly along the ring as it moves through the environment. The circular shape suggests robust encoding of continuous spatial factors, reflecting an internally consistent strategy rather than rote stimulus-response mappings.