EZBlender: Efficient 3D Editing with Plan-and-ReAct Agent

TL;DR

EZBlender addresses the efficiency–precision trade-off in 3D editing by integrating a Plan-and-ReAct architecture with edge autonomy, enabling semantic planning and domain-specific execution to run in parallel. The Planner decomposes user intent into domain-aware directives, while specialized Sub-Agents ground these into executable Blender constraints with bounded propose–verify–refine loops, reducing bottlenecks and latency. Through a dedicated BlenderGym-based benchmark and CLIP-based evaluations across text and visual prompts, EZBlender demonstrates competitive editing fidelity and significantly improved responsiveness—up to $7$ times faster and with substantially lower token usage—compared to state-of-the-art Blender-based agents. The results show robust text–visual alignment and effective multi-task editing, highlighting practical benefits for AI-assisted 3D workflows and offering a scalable design for domain-adapted, efficient agentic systems.

Abstract

As a cornerstone of the modern digital economy, 3D modeling and rendering demand substantial resources and manual effort when scene editing is performed in the traditional manner. Despite recent progress in VLM-based agents for 3D editing, the fundamental trade-off between editing precision and agent responsiveness remains unresolved. To overcome these limitations, we present EZBlender, a Blender agent with a hybrid framework that combines planning-based task decomposition and reactive local autonomy for efficient human AI collaboration and semantically faithful 3D editing. Specifically, this unexplored Plan-and-ReAct design not only preserves editing quality but also significantly reduces latency and computational cost. To further validate the efficiency and effectiveness of the proposed edge-autonomy architecture, we construct a dedicated multi-tasking benchmark that has not been systematically investigated in prior research. In addition, we provide a comprehensive analysis of language model preference, system responsiveness, and economic efficiency.

Figures (7)

• Figure 1: The design concept of 3D editing agents. (a) The leap-tune design implemented in BlenderAlchemyhuang2024blenderalchemy; (b) The proposed Plan-and-ReAct agent design.
• Figure 2: Agent design comparison. (a) The mainstream ReAct yao2023react agent design; (b) The mainstream Plan-and-Execute agent design; (c) The proposed Plan-and-ReAct agent design optimized for 3D editing work.
• Figure 3: Proposed EZBlender framework. The planner agent takes user input and decomposes it into several sub-tasks with templated reasoning. Specialized sub-agents then generate the corresponding scene components and refine them through edge autonomy. The system outputs the final response once all components are completed.
• Figure 4: Visualization results of our method.(Left.) Text-prompt-driven editing, where our agent applies textual instructions to manipulate 3D-rendered avatars and objects, including attribute modifications (e.g., skin color, accessories, identity), stylistic transformations (e.g., cartoon, pixel-art), and scene/background changes. (Right.) Visual-prompt-driven editing, where reference images guide transformations, enabling shape transfer, illumination adjustment, structural modifications, and scene-level re-rendering (e.g., architectural edits, object replacement).
• Figure 5: Multi-Tasking Benchmark. Five scenarios (50 episodes in total) with editable objects are selected, each paired with a randomly generated prompt to evaluate agent performance on multi-task editing within a single trial. Each trial involves coordinated edits of shape keys, materials, background, lighting, and camera settings according to the prompt.