Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ComfySearch: Autonomous Exploration and Reasoning for ComfyUI Workflows

Jinwei Su, Qizhen Lan, Zeyu Wang, Yinghui Xia, Hairu Wen, Yiqun Duan, Xi Xiao, Tianyu Shi, Yang Jingsong, Lewei He

TL;DR

ComfySearch introduces an execution-grounded, reasoning-as-action framework for autonomous ComfyUI workflow construction. By coupling state-aware online validation with in-place repair and entropy-adaptive branching, it robustly builds long-horizon, typified node graphs that remain executable at every prefix. The method is trained with a two-stage pipeline—supervised warmup and tool-mediated RL using GRPO—and guided by consistency-filtered data and validator-backed traces. Evaluations on ComfyBench and GenEval show substantial gains in executability, task success, and downstream compositional accuracy, highlighting the practical impact of execution-grounded exploration for modular creative pipelines.

Abstract

AI-generated content has progressed from monolithic models to modular workflows, especially on platforms like ComfyUI, allowing users to customize complex creative pipelines. However, the large number of components in ComfyUI and the difficulty of maintaining long-horizon structural consistency under strict graph constraints frequently lead to low pass rates and workflows of limited quality. To tackle these limitations, we present ComfySearch, an agentic framework that can effectively explore the component space and generate functional ComfyUI pipelines via validation-guided workflow construction. Experiments demonstrate that ComfySearch substantially outperforms existing methods on complex and creative tasks, achieving higher executability (pass) rates, higher solution rates, and stronger generalization.

ComfySearch: Autonomous Exploration and Reasoning for ComfyUI Workflows

TL;DR

ComfySearch introduces an execution-grounded, reasoning-as-action framework for autonomous ComfyUI workflow construction. By coupling state-aware online validation with in-place repair and entropy-adaptive branching, it robustly builds long-horizon, typified node graphs that remain executable at every prefix. The method is trained with a two-stage pipeline—supervised warmup and tool-mediated RL using GRPO—and guided by consistency-filtered data and validator-backed traces. Evaluations on ComfyBench and GenEval show substantial gains in executability, task success, and downstream compositional accuracy, highlighting the practical impact of execution-grounded exploration for modular creative pipelines.

Abstract

AI-generated content has progressed from monolithic models to modular workflows, especially on platforms like ComfyUI, allowing users to customize complex creative pipelines. However, the large number of components in ComfyUI and the difficulty of maintaining long-horizon structural consistency under strict graph constraints frequently lead to low pass rates and workflows of limited quality. To tackle these limitations, we present ComfySearch, an agentic framework that can effectively explore the component space and generate functional ComfyUI pipelines via validation-guided workflow construction. Experiments demonstrate that ComfySearch substantially outperforms existing methods on complex and creative tasks, achieving higher executability (pass) rates, higher solution rates, and stronger generalization.
Paper Structure (42 sections, 13 equations, 5 figures, 5 tables)

This paper contains 42 sections, 13 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Method
  4. Dataset Construction
  5. Consistency-filtered pairing.
  6. Reasoning traces with validator feedback.
  7. Reasoning-As-Action Graph-Editing MDP
  8. Workflow representation.
  9. State and actions.
  10. Tool-mediated transition.
  11. State-Aware Validation and In-Place Repair (C1)
  12. State-aware validator.
  13. In-place repair with diagnostics.
  14. Validated-prefix property.
  15. Entropy-Adaptive Branching Under Uncertainty (C2)
  16. ...and 27 more sections

Figures (5)

  • Figure 1: One-shot workflow planning generates the entire graph at once and is brittle, whereas reasoning-as-action generation (ComfySearch) incrementally builds and validates each edit, ensuring robust and executable workflows.
  • Figure 2: Overview of the ComfySearch framework. We formulate workflow generation as a Markov Decision Process(§\ref{['sec:method:mdp']}). (a) State-aware Validation(§\ref{['sec:method:c1']}): Implements real-time verification and repair to ensure online structural correctness (C1). (b) Entropy Rollout Branching(§\ref{['sec:method:c2']}): Employs selective exploration at high-uncertainty points to navigate long-horizon decision branching (C2).
  • Figure 3: Examples generated by ComfySearch.
  • Figure 4: Qualitative results showing the performance of our method on various tasks.
  • Figure 5: Qualitative comparison between our method and ComfyMind