LogicEnvGen: Task-Logic Driven Generation of Diverse Simulated Environments for Embodied AI

TL;DR

LogicEnvGen introduces a top-down, LLM-driven framework to generate logically diverse simulated environments for embodied AI, addressing the gap in prior work that emphasized visual realism over task-level diversity. The method derives behavior plans with decision trees, synthesizes non-redundant logical trajectories, and instantiates physically plausible environments through floor-plan design, asset selection via CLIP, and CSP-based object arrangement with a Z3 solver. To evaluate the utility of logically diverse environments, the authors propose LogicEnvEval, a benchmark with 25 household tasks and four agent policies, interrogated by four metrics: Physics Pass Rate, Logic Coverage, Scenario Validity Rate, and Fault Detection Rate. Experiments show that LogicEnvGen achieves higher logical coverage (1.04–2.61×) and substantially improves fault detection (4–68%) while maintaining physical plausibility, outperforming Holodeck, CoT, and IFG baselines across three LLMs. The work advances embodied AI evaluation by enabling systematic, scalable, and reinforcement-friendly testing across diverse task scenarios, with practical impact on improving agent adaptability and planning robustness.

Abstract

Simulated environments play an essential role in embodied AI, functionally analogous to test cases in software engineering. However, existing environment generation methods often emphasize visual realism (e.g., object diversity and layout coherence), overlooking a crucial aspect: logical diversity from the testing perspective. This limits the comprehensive evaluation of agent adaptability and planning robustness in distinct simulated environments. To bridge this gap, we propose LogicEnvGen, a novel method driven by Large Language Models (LLMs) that adopts a top-down paradigm to generate logically diverse simulated environments as test cases for agents. Given an agent task, LogicEnvGen first analyzes its execution logic to construct decision-tree-structured behavior plans and then synthesizes a set of logical trajectories. Subsequently, it adopts a heuristic algorithm to refine the trajectory set, reducing redundant simulation. For each logical trajectory, which represents a potential task situation, LogicEnvGen correspondingly instantiates a concrete environment. Notably, it employs constraint solving for physical plausibility. Furthermore, we introduce LogicEnvEval, a novel benchmark comprising four quantitative metrics for environment evaluation. Experimental results verify the lack of logical diversity in baselines and demonstrate that LogicEnvGen achieves 1.04-2.61x greater diversity, significantly improving the performance in revealing agent faults by 4.00%-68.00%.

Figures (15)

• Figure 1: Simulated environments generated by Holodeck yang2024holodeck and LogicEnvGen. In contrast, LogicEnvGen generates more logically diverse test cases based on the task description, enabling more comprehensive simulation.
• Figure 2: Overview. Phase ①: Behavior Plan Derivation, decomposes the given task into independent subtasks, identifies uncertain environment factors impacting subtask execution, and generates decision-tree-structured behavior plan for each subtask. Phase ②: Logical Trajectory Collection, traverses and combines decision paths across trees to synthesize distinct logical trajectories for the entire task, each representing a potential task situation. Phase ③: Simulated Environment Construction, instantiates a concrete and physically plausible environment for each situation through three stages: floor plan design, environment object selection and arrangement.
• Figure 3: Algorithm example. Logical trajectories reduce from 12 to 4 (3 blue, 1 purple).
• Figure 4: LogicEnvEval Benchmark. (a) Task composition. SN: number of subtasks. TN: number of task trajectories (minimal). (b) Environment type distribution of all subtasks. (c) Action step distribution of correct policies. (d) Action type diversity in correct policies.
• Figure 5: Results of ablation study on the constraint-based arrangement solving (CAS).