BideDPO: Conditional Image Generation with Simultaneous Text and Condition Alignment

TL;DR

This work tackles the challenge of aligning both text prompts and conditioning inputs in conditional image generation, identifying input-level and model-bias conflicts. It introduces BideDPO, a bidirectionally decoupled Direct Preference Optimization framework that separates text and condition learning signals and uses Adaptive Loss Balancing to prevent gradient entanglement. An automated data pipeline generates disentangled, conflict-aware preference data, and an iterative optimization loop refines both data and the model, yielding strong improvements on the DualAlign and COCO benchmarks, including style-conditioned cases. By formulating separate reward-based objectives for text and condition adherence and validating with multiple evaluators, the approach significantly enhances both textual fidelity and conditioning fidelity in multi-constraint generation with practical impact for advanced, controllable image synthesis.

Abstract

Conditional image generation enhances text-to-image synthesis with structural, spatial, or stylistic priors, but current methods face challenges in handling conflicts between sources. These include 1) input-level conflicts, where the conditioning image contradicts the text prompt, and 2) model-bias conflicts, where generative biases disrupt alignment even when conditions match the text. Addressing these conflicts requires nuanced solutions, which standard supervised fine-tuning struggles to provide. Preference-based optimization techniques like Direct Preference Optimization (DPO) show promise but are limited by gradient entanglement between text and condition signals and lack disentangled training data for multi-constraint tasks. To overcome this, we propose a bidirectionally decoupled DPO framework (BideDPO). Our method creates two disentangled preference pairs-one for the condition and one for the text-to reduce gradient entanglement. The influence of pairs is managed using an Adaptive Loss Balancing strategy for balanced optimization. We introduce an automated data pipeline to sample model outputs and generate conflict-aware data. This process is embedded in an iterative optimization strategy that refines both the model and the data. We construct a DualAlign benchmark to evaluate conflict resolution between text and condition. Experiments show BideDPO significantly improves text success rates (e.g., +35%) and condition adherence. We also validate our approach using the COCO dataset. Project Pages: https://limuloo.github.io/BideDPO/.

Figures (12)

• Figure 1: Qualitative comparison on cases with conflicting text and condition. We first introduce two conflicts between the text prompt and conditioning input: (a) Input Level Conflict and (b) Model Bias Conflict, which hinder model controllability. We then propose a solution that resolves both, generating images that satisfy both the text and the condition. "default cond." means using its default condition constraint scale, while "low cond." means using a lower condition constraint scale. (c) Our method also enhances the alignment between text and abstract conditions such as style condition, and supports generation with multiple conditions combined with text prompts.
• Figure 2: Comparison between vanilla DPO and our bidirectionally decoupled DPO for conditional image generation. a) Vanilla DPO uses coupled preference pairs, so its gradients can become ambiguous or even vanish when text and condition are not aligned together. b) BideDPO separates the learning signals for text and condition and adaptively balance them. This provides clear, adaptive gradients for each requirement, allowing the model to achieve better multi-constraint alignment.
• Figure 3: The Automated Disentangled, Conflict-Aware Preference Data Generation Pipeline.
• Figure 4: Iterative Optimization Strategy. We start with an initial generator ($G^0$) that produces training data via our automated pipeline (Fig. \ref{['fig:data_pipe']}). Training with BideDPO already improves the model ($G^1$), while repeating the process with the updated generator yields higher-quality data and further gains, forming a self-reinforcing loop where both data and model improve progressively.
• Figure 5: Visual comparison for conditional image generation on the DualAlign Benchmark. We evaluate three common conditioning modalities: depth, Canny, and soft edge. Our method improves adherence to both the text prompt and the spatial conditioning. Please zoom in for details.