Text-guided Controllable Diffusion for Realistic Camouflage Images Generation

TL;DR

CT-CIG introduces a text-guided controllable diffusion framework for realistic camouflage image generation. It combines a Camouflage-Revealing Dialogue Mechanism (CRDM) to generate semantically aligned prompts with a lightweight controller, a Frequency Interaction Refinement Module (FIRM) for high-frequency texture, and Cross Normalization (CN) for stable conditioning. Finetuning a diffusion backbone on camouflage data with perceptual LPIPS loss and text-guided prompts yields photorealistic, logically plausible camouflage, validated by FID/KID/CLIPScore on the LAKE-RED dataset. The approach achieves a favorable balance between image fidelity, semantic alignment, and efficiency, and suggests a scalable direction for text-guided camouflage and other texture-rich, context-aware generation tasks.

Abstract

Camouflage Images Generation (CIG) is an emerging research area that focuses on synthesizing images in which objects are harmoniously blended and exhibit high visual consistency with their surroundings. Existing methods perform CIG by either fusing objects into specific backgrounds or outpainting the surroundings via foreground object-guided diffusion. However, they often fail to obtain natural results because they overlook the logical relationship between camouflaged objects and background environments. To address this issue, we propose CT-CIG, a Controllable Text-guided Camouflage Images Generation method that produces realistic and logically plausible camouflage images. Leveraging Large Visual Language Models (VLM), we design a Camouflage-Revealing Dialogue Mechanism (CRDM) to annotate existing camouflage datasets with high-quality text prompts. Subsequently, the constructed image-prompt pairs are utilized to finetune Stable Diffusion, incorporating a lightweight controller to guide the location and shape of camouflaged objects for enhanced camouflage scene fitness. Moreover, we design a Frequency Interaction Refinement Module (FIRM) to capture high-frequency texture features, facilitating the learning of complex camouflage patterns. Extensive experiments, including CLIPScore evaluation and camouflage effectiveness assessment, demonstrate the semantic alignment of our generated text prompts and CT-CIG's ability to produce photorealistic camouflage images.

Figures (13)

• Figure 1: Example images generated by CT-CIG, which proves its ability to handle objects of different attributes.
• Figure 2: Overall framework of our proposed CT-CIG, which performs camouflage generation via three steps. (1) Extracting features of input images and masks through VAE and controller, followed by control augmentation through FIRM and CN. (2) Generating text prompts from the VLM through CRDM and using the CLIP encoder to obtain embeddings. (3) Performing controllable stable diffusion and generating results.
• Figure 3: Intra details of the Camouflage-Revealing Dialogue Mechanism. VLMs should obey the rules in system messages to produce answers that meet our requirements. Different queries are designed for camouflage images and non-camouflage images to guide them to generate camouflage-originated responses.
• Figure 4: Frequency Interaction Refinement Module.
• Figure 5: Results of generated images with different methods. The first two columns show the real images and paired masks in the COD datasets. Backgrounds in column 3 are randomly selected. Methods in columns 5-8 require text prompts as a condition. All methods take masks as shape guidance except LDM-T2I.