Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Aligning Agentic World Models via Knowledgeable Experience Learning

Baochang Ren, Yunzhi Yao, Rui Sun, Shuofei Qiao, Ningyu Zhang, Huajun Chen

TL;DR

WorldMind tackles the gap between semantic reasoning in large models and physical grounding by externally encoding environmental dynamics into a World Knowledge Repository consisting of Process Experience and Goal Experience. Grounding occurs in a training-free, online fashion via Predictive Coding, where prediction errors refine physical feasibility and successful trajectories shape procedural heuristics. The framework defines a WK-MDP with priors W_p and W_g to constrain policy search and enable constrained simulation during inference. Empirical results on EB-ALFRED, EB-Habitat, and cross-domain Embodied Web Agent benchmarks show reduced physical hallucinations, improved strict task completion (SR), and enhanced subgoal correctness (GC), with strong cross-model transferability of the learned world knowledge. These findings suggest that externalizing environmental dynamics as symbolic priors yields robust, reusable knowledge for generalist embodied agents across tasks and embodiments.

Abstract

Current Large Language Models (LLMs) exhibit a critical modal disconnect: they possess vast semantic knowledge but lack the procedural grounding to respect the immutable laws of the physical world. Consequently, while these agents implicitly function as world models, their simulations often suffer from physical hallucinations-generating plans that are logically sound but physically unexecutable. Existing alignment strategies predominantly rely on resource-intensive training or fine-tuning, which attempt to compress dynamic environmental rules into static model parameters. However, such parametric encapsulation is inherently rigid, struggling to adapt to the open-ended variability of physical dynamics without continuous, costly retraining. To bridge this gap, we introduce WorldMind, a framework that autonomously constructs a symbolic World Knowledge Repository by synthesizing environmental feedback. Specifically, it unifies Process Experience to enforce physical feasibility via prediction errors and Goal Experience to guide task optimality through successful trajectories. Experiments on EB-ALFRED and EB-Habitat demonstrate that WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.

Aligning Agentic World Models via Knowledgeable Experience Learning

TL;DR

WorldMind tackles the gap between semantic reasoning in large models and physical grounding by externally encoding environmental dynamics into a World Knowledge Repository consisting of Process Experience and Goal Experience. Grounding occurs in a training-free, online fashion via Predictive Coding, where prediction errors refine physical feasibility and successful trajectories shape procedural heuristics. The framework defines a WK-MDP with priors W_p and W_g to constrain policy search and enable constrained simulation during inference. Empirical results on EB-ALFRED, EB-Habitat, and cross-domain Embodied Web Agent benchmarks show reduced physical hallucinations, improved strict task completion (SR), and enhanced subgoal correctness (GC), with strong cross-model transferability of the learned world knowledge. These findings suggest that externalizing environmental dynamics as symbolic priors yields robust, reusable knowledge for generalist embodied agents across tasks and embodiments.

Abstract

Current Large Language Models (LLMs) exhibit a critical modal disconnect: they possess vast semantic knowledge but lack the procedural grounding to respect the immutable laws of the physical world. Consequently, while these agents implicitly function as world models, their simulations often suffer from physical hallucinations-generating plans that are logically sound but physically unexecutable. Existing alignment strategies predominantly rely on resource-intensive training or fine-tuning, which attempt to compress dynamic environmental rules into static model parameters. However, such parametric encapsulation is inherently rigid, struggling to adapt to the open-ended variability of physical dynamics without continuous, costly retraining. To bridge this gap, we introduce WorldMind, a framework that autonomously constructs a symbolic World Knowledge Repository by synthesizing environmental feedback. Specifically, it unifies Process Experience to enforce physical feasibility via prediction errors and Goal Experience to guide task optimality through successful trajectories. Experiments on EB-ALFRED and EB-Habitat demonstrate that WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.
Paper Structure (33 sections, 6 equations, 4 figures, 4 tables)

This paper contains 33 sections, 6 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. The WorldMind Framework
  4. Overview and Problem Formulation
  5. Building World Knowledge Repository
  6. Process Experience Construction.
  7. Goal Experience Construction.
  8. Inference via Constrained Simulation
  9. Experiment
  10. Experimental Settings.
  11. Datasets and Metrics.
  12. Models and Baselines.
  13. Main Results
  14. WorldMind consistently outperforms baselines in strict task completion.
  15. WorldMind achieves superior procedural correctness.
  16. ...and 18 more sections

Figures (4)

  • Figure 1: Conceptual Illustration of Experiential Alignment. The agent aligns its internal world model via Process Experience and Goal Experience.
  • Figure 2: Overview of the WorldMind Framework. The agent autonomously constructs a World Knowledge Repository (WKR) by unifying Process Experience (from prediction errors) and Goal Experience (from successful trajectories) to guide grounded simulation.
  • Figure 3: Experimental Results and Analysis. (\ref{['fig:transfer_exp']}) Comparison of experience transfer capabilities between the two models. (\ref{['fig:web_accuracy']}) Performance comparison on the Embodied Web Agent task, reporting accuracy metrics for both GPT-3.5-turbo and GPT-4.1-mini. (\ref{['fig:web_error']}) Comparative error distribution analysis for both models in the same environment.
  • Figure 4: Error Analysis in EB-ALFRED and EB-Habitat. Comparison of error distributions between the ReAct baseline and WorldMind. Failures are categorized into three types: Invalid Actions, Timeout, and Wrong Termination.