OSCAR: Orthogonal Stochastic Control for Alignment-Respecting Diversity in Flow Matching

TL;DR

This work addresses the lack of semantic diversity in flow-based text-to-image generation by introducing OSCAR, a training-free, inference-time control that makes the sampling dynamics diversity-aware. It combines a deterministic, endpoint-aware feature-volume guidance with time-scheduled orthogonal stochastic perturbations, ensuring the diversity push remains orthogonal to the base flow to preserve fidelity. The method provides theoretical guarantees of monotone volume increase and approximate marginal preservation, and empirically delivers improved diversity metrics (e.g., Vendi Score, mode coverage, and DIM/CIM) without sacrificing image quality across multiple CFG scales and data settings. OSCAR operates without retraining, integrates with standard flow-matching solvers, and demonstrates robustness to sampling budgets and hyperparameters, offering a practical pathway to richer, more controllable generative outputs.

Abstract

Flow-based text-to-image models follow deterministic trajectories, forcing users to repeatedly sample to discover diverse modes, which is a costly and inefficient process. We present a training-free, inference-time control mechanism that makes the flow itself diversity-aware. Our method simultaneously encourages lateral spread among trajectories via a feature-space objective and reintroduces uncertainty through a time-scheduled stochastic perturbation. Crucially, this perturbation is projected to be orthogonal to the generation flow, a geometric constraint that allows it to boost variation without degrading image details or prompt fidelity. Our procedure requires no retraining or modification to the base sampler and is compatible with common flow-matching solvers. Theoretically, our method is shown to monotonically increase a volume surrogate while, due to its geometric constraints, approximately preserving the marginal distribution. This provides a principled explanation for why generation quality is robustly maintained. Empirically, across multiple text-to-image settings under fixed sampling budgets, our method consistently improves diversity metrics such as the Vendi Score and Brisque over strong baselines, while upholding image quality and alignment.

Figures (24)

• Figure 1: Left: A conceptual comparison of the generation process. (A) The standard flow matching shows independent trajectories collapsing to similar modes. (B)OSCAR (our method) introduces an orthogonal control mechanism that forces the interacting trajectories to diverge and cover a wider semantic space. Right: A qualitative comparison of generated images against strong baselines. The combined results illustrate that our method significantly increases the diversity of generations while retaining high output quality.
• Figure 2: 2D toy visualization of diversity enhancement on a 3x3 GMM. The top row shows the standard Flow Matching baseline, while the bottom row shows our method. Columns are snapshots at early, middle, and final generation steps. Black "+" markers indicate the GMM means, blue dots are the shared initial particles, and orange "x" markers are the particle locations at the given step. Our method yields more uniform within-mode coverage and better-separated modes.
• Figure 3: PRD curves on the 'truck' concept, comparing methods across different CFG levels. Curves shifted toward the top-right indicate a superior precision-recall trade-off.
• Figure 4: (a) Mode Coverage vs. Threshold ($\tau$) for the 'truck' concept, comparing our method against baselines at three fixed CFG levels. Higher curves indicate a larger fraction of real-data clusters are covered, signifying superior mode discovery. Shaded bands denote the $\pm 1$ standard deviation across multiple seeds and prompts. (b) Normalized entropy for the 'truck' concept, comparing methods at different CFG levels. Higher bars indicate a more uniform distribution of samples across discovered modes, signifying less redundancy. Error bars denote the $\pm 1$ standard deviation.
• Figure 5: Evaluation of attribute-level diversity and generalization on the bus concept. DIM quantifies the balance of attributes generated from coarse prompts (a score closer to 0 indicates better balance), while CIM assesses the ability to follow explicit attribute requests (a higher score is better).

Theorems & Definitions (7)

• Lemma 1: Pullback identity
• Theorem 1: Expected energy descent & marginal quality preservation
• proof : Sketch
• Corollary 1: Global deviation of the alignment coordinate
• Remark 1: Robustness to schedule and scale
• Theorem 2: Energy $\downarrow$ $\Longleftrightarrow$ volume $\uparrow$ $\Longrightarrow$ diversity $\uparrow$
• Remark 2: Discretization and redundancy-aware reweighting