Language-Grounded Multi-Domain Image Translation via Semantic Difference Guidance

TL;DR

This work tackles language-grounded multi-domain image-to-image translation while preserving image structure. It introduces LACE, a diffusion-based framework comprising a Global-Local Image Prompt Adapter (GLIP-Adapter) for image-conditioned prompts and a Multi-Domain Control Guidance (MCG) that grounds semantic prompt differences into noise-space translations, enabling compositional, per-attribute control. Through experiments on CelebA(Dialog) and BDD100K, LACE achieves high visual fidelity, robust structure preservation, and interpretable domain-specific control, surpassing prior baselines and showing strong performance across 1–3 attribute edits. The approach advances cross-modal content generation by coupling language semantics with controllable, structure-aware diffusion processes, while also acknowledging computational costs and the need for careful consideration of potential misuse.

Abstract

Multi-domain image-to-image translation re quires grounding semantic differences ex pressed in natural language prompts into corresponding visual transformations, while preserving unrelated structural and seman tic content. Existing methods struggle to maintain structural integrity and provide fine grained, attribute-specific control, especially when multiple domains are involved. We propose LACE (Language-grounded Attribute Controllable Translation), built on two compo nents: (1) a GLIP-Adapter that fuses global semantics with local structural features to pre serve consistency, and (2) a Multi-Domain Control Guidance mechanism that explicitly grounds the semantic delta between source and target prompts into per-attribute translation vec tors, aligning linguistic semantics with domain level visual changes. Together, these modules enable compositional multi-domain control with independent strength modulation for each attribute. Experiments on CelebA(Dialog) and BDD100K demonstrate that LACE achieves high visual fidelity, structural preservation, and interpretable domain-specific control, surpass ing prior baselines. This positions LACE as a cross-modal content generation framework bridging language semantics and controllable visual translation.

Figures (8)

• Figure 1: Multi-domain translation results with progressive attribute modifications. Our method enables compositional editing of multiple attributes while preserving non-target regions and structural consistency.
• Figure 2: Overview of our proposed multi-domain I2I translation framework. The GLIP-Adapter projects global (CLIP) and local (DINOv2) features into a linear projection, which together with the source text prompt is used to train the U-Net’s cross-attention layers. At inference, the MCG module leverages source–target prompt differences to guide controllable multi-attribute translation.
• Figure 3: Qualitative evaluation for multi-domain image-to-image translation methods on CelebA.
• Figure 4: Qualitative evaluation for multi-domain image-to-image translation methods on BDD100K.
• Figure 5: The ablation study of multi-domain image-to-image translation methods