Agentic Retoucher for Text-To-Image Generation

TL;DR

This work tackles persistent localized distortions in text-to-image diffusion outputs by reframing post-generation editing as a perception–reasoning–action loop. It introduces Agentic Retoucher, a three-agent system that localizes distortions via a context-aware saliency predictor, performs human-aligned reasoning with progressive preference optimization, and executes adaptive, region-specific inpainting through a modular toolset. To enable supervised learning and evaluation of fine-grained artifacts, the GenBlemish-27K dataset provides pixel-level masks and natural-language descriptions across 12 artifact categories. Empirical results show consistent improvements in local perceptual quality, distortion localization, and human preference alignment over state-of-the-art baselines, establishing a self-corrective paradigm for reliable T2I generation. The approach promises practical impact in industrial and creative workflows by enabling autonomous, interpretable refinements without full-image re-generation.

Abstract

Text-to-image (T2I) diffusion models such as SDXL and FLUX have achieved impressive photorealism, yet small-scale distortions remain pervasive in limbs, face, text and so on. Existing refinement approaches either perform costly iterative re-generation or rely on vision-language models (VLMs) with weak spatial grounding, leading to semantic drift and unreliable local edits. To close this gap, we propose Agentic Retoucher, a hierarchical decision-driven framework that reformulates post-generation correction as a human-like perception-reasoning-action loop. Specifically, we design (1) a perception agent that learns contextual saliency for fine-grained distortion localization under text-image consistency cues, (2) a reasoning agent that performs human-aligned inferential diagnosis via progressive preference alignment, and (3) an action agent that adaptively plans localized inpainting guided by user preference. This design integrates perceptual evidence, linguistic reasoning, and controllable correction into a unified, self-corrective decision process. To enable fine-grained supervision and quantitative evaluation, we further construct GenBlemish-27K, a dataset of 6K T2I images with 27K annotated artifact regions across 12 categories. Extensive experiments demonstrate that Agentic Retoucher consistently outperforms state-of-the-art methods in perceptual quality, distortion localization and human preference alignment, establishing a new paradigm for self-corrective and perceptually reliable T2I generation.

Figures (5)

• Figure 1: Left: Existing VLMs hallucinate and fail to localize distortions in AIGC-images, even with explicit region cues, whereas our method accurately localizes distorted regions and provides reasonable diagnoses. Right: Each before-after pair shows the distorted image and the result refined by our Agentic Retoucher, including diverse distortion artifacts across text, hand, face, and interaction.
• Figure 2: Overview of GenBlemish-27K. The figure illustrates (a) the dual-layer distortion taxonomy with six high-level dimensions and twelve fine-grained categories, (b) the distribution of localized distortion types, (c) the human-AI collaborative annotation pipeline, and (d) representative formatted samples with pixel-level masks and textual descriptions, highlighting how GenBlemish-27K enables fine-grained localization and reasoning over diverse text-to-image distortions.
• Figure 3: Overview of the proposed Agentic Retoucher. The framework operates as a perception-reasoning-action loop for post-generation correction in AIGC. The Perception Agent localizes context-dependent distortions via cross-modal saliency prediction, the Reasoning Agent performs human-aligned diagnosis through iterative reasoning, and the Action Agent executes adaptive localized inpainting guided by reasoning outputs, forming a closed-loop self-corrective process.
• Figure 4: Qualitative comparison of retouching results across diverse prompts. White bounding boxes indicate zoomed-in fine-grained regions. Agentic Retoucher retouches local distortions and implausibilities while maintaining global visual harmony, outperforming both VLM-based and mask-based baselines. More qualitative results are included in the supplementary material.
• Figure 5: Qualitative visualization of saliency prediction. Our method yields sharper, context-aware localization than RichHF and GLM4.1V.