Steve-Evolving: Open-World Embodied Self-Evolution via Fine-Grained Diagnosis and Dual-Track Knowledge Distillation

Abstract

Open-world embodied agents must solve long-horizon tasks where the main bottleneck is not single-step planning quality but how interaction experience is organized and evolved. To this end, we present Steve-Evolving, a non-parametric self-evolving framework that tightly couples fine-grained execution diagnosis with dual-track knowledge distillation in a closed loop. The method follows three phases: Experience Anchoring, Experience Distillation, and Knowledge-Driven Closed-Loop Control. In detail, Experience Anchoring solidifies each subgoal attempt into a structured experience tuple with a fixed schema (pre-state, action, diagnosis-result, and post-state) and organizes it in a three-tier experience space with multi-dimensional indices (e.g., condition signatures, spatial hashing, and semantic tags) plus rolling summarization for efficient and auditable recall. To ensure sufficient information density for attribution, the execution layer provides compositional diagnosis signals beyond binary outcomes, including state-difference summaries, enumerated failure causes, continuous indicators, and stagnation/loop detection. Moreover, successful trajectories of Experience Distillation are generalized into reusable skills with explicit preconditions and verification criteria, while failures are distilled into executable guardrails that capture root causes and forbid risky operations at both subgoal and task granularities. Besides, Knowledge-Driven Closed-Loop Control retrieved skills and guardrails are injected into an LLM planner, and diagnosis-triggered local replanning updates the active constraints online, forming a continual evolution process without any model parameter updates. Experiments on the long-horizon suite of Minecraft MCU demonstrate consistent improvements over static-retrieval baselines.

Figures (7)

• Figure 1: Diagnosis-triggered knowledge-driven planning (horizontal layout). A failed subgoal execution is first analyzed by fine-grained diagnostic signals (e.g., loop detection and unsafe-area failures), then distilled into two reusable knowledge types: guardrails that forbid risky actions under specified triggers, and skills that summarize successful procedures with explicit preconditions and verification. The retrieved guardrails and skills are injected into the LLM planner to guide safer planning and to trigger local replanning when failures recur, forming a closed-loop of diagnosis, distillation, and knowledge-driven control.
• Figure 2: Minecraft technology progression from wooden to diamond tiers, illustrating the hierarchy of tools, materials, and equipment used to construct long-horizon tasks for embodied agent evaluation.
• Figure 3: Overview of the proposed experience evolution framework for open-world embodied agents. Embodied interactions are first diagnosed through fine-grained execution signals and recorded as structured experience documents. A dual-track distillation mechanism then extracts reusable skills from successful trajectories and guardrails from failure cases. The distilled knowledge is injected into the LLM planner to guide decision-making and trigger replanning when failures accumulate, forming a closed-loop process of experience accumulation and knowledge-driven control.
• Figure 4: visualization of Ablation results.
• Figure 5: (a) Diverse block types form an explorable, harvestable, and modifiable world. (b) The crafting interface illustrates recipe-based item composition and tool progression. (c) Survival mode introduces continuous survival pressure from health, hunger, and environmental hazards. (d) The day--night cycle changes visibility and threats, making exploration and shelter building more strategic.