Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PromptIQ: Who Cares About Prompts? Let System Handle It -- A Component-Aware Framework for T2I Generation

Nisan Chhetri, Arpan Sainju

TL;DR

PromptIQ tackles the difficulty of producing high-quality T2I images without expert prompt engineering by introducing a component-aware evaluation metric called CAS that detects structural inaccuracies CLIP overlooks. The framework automatically refines prompts and images through a five-phase loop including SAM-based subject isolation, component segmentation, and BLIP-SBERT based similarity to a predefined true_caption_list, with ChatGPT-driven prompt refinement guiding iterations. Experimental results across four subjects show CAS differentiates flawed from well-structured images and outperforms CLIP in structural assessment, leading to more reliable image quality. The approach makes T2I more accessible for non-experts and holds promise for applications where component-level accuracy matters, with future work extending to multiple diffusion models and richer user control.

Abstract

Generating high-quality images without prompt engineering expertise remains a challenge for text-to-image (T2I) models, which often misinterpret poorly structured prompts, leading to distortions and misalignments. While humans easily recognize these flaws, metrics like CLIP fail to capture structural inconsistencies, exposing a key limitation in current evaluation methods. To address this, we introduce PromptIQ, an automated framework that refines prompts and assesses image quality using our novel Component-Aware Similarity (CAS) metric, which detects and penalizes structural errors. Unlike conventional methods, PromptIQ iteratively generates and evaluates images until the user is satisfied, eliminating trial-and-error prompt tuning. Our results show that PromptIQ significantly improves generation quality and evaluation accuracy, making T2I models more accessible for users with little to no prompt engineering expertise.

PromptIQ: Who Cares About Prompts? Let System Handle It -- A Component-Aware Framework for T2I Generation

TL;DR

PromptIQ tackles the difficulty of producing high-quality T2I images without expert prompt engineering by introducing a component-aware evaluation metric called CAS that detects structural inaccuracies CLIP overlooks. The framework automatically refines prompts and images through a five-phase loop including SAM-based subject isolation, component segmentation, and BLIP-SBERT based similarity to a predefined true_caption_list, with ChatGPT-driven prompt refinement guiding iterations. Experimental results across four subjects show CAS differentiates flawed from well-structured images and outperforms CLIP in structural assessment, leading to more reliable image quality. The approach makes T2I more accessible for non-experts and holds promise for applications where component-level accuracy matters, with future work extending to multiple diffusion models and richer user control.

Abstract

Generating high-quality images without prompt engineering expertise remains a challenge for text-to-image (T2I) models, which often misinterpret poorly structured prompts, leading to distortions and misalignments. While humans easily recognize these flaws, metrics like CLIP fail to capture structural inconsistencies, exposing a key limitation in current evaluation methods. To address this, we introduce PromptIQ, an automated framework that refines prompts and assesses image quality using our novel Component-Aware Similarity (CAS) metric, which detects and penalizes structural errors. Unlike conventional methods, PromptIQ iteratively generates and evaluates images until the user is satisfied, eliminating trial-and-error prompt tuning. Our results show that PromptIQ significantly improves generation quality and evaluation accuracy, making T2I models more accessible for users with little to no prompt engineering expertise.
Paper Structure (6 sections, 2 figures, 2 algorithms)

This paper contains 6 sections, 2 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. PromptIQ Framework
  3. Experiments
  4. Experimental Setup
  5. Results and Analysis
  6. Conclusion and Future work

Figures (2)

  • Figure 1: Overview of our automated component-aware framework - PromptIQ, which iteratively refines generated images through a five-phase process, incorporating user feedback to ensure structural accuracy and high-quality results—without the need for prompt engineering.
  • Figure 2: Comparison of CLIP and CAS scores across four subjects—car, bus, truck, and bicycle—before and after prompt modification. CAS effectively differentiates structurally flawed images from well-formed ones, while CLIP remains largely insensitive to these variations.