Cross-fluctuation phase transitions reveal sampling dynamics in diffusion models

TL;DR

This work introduces cross-fluctuation mergers, a centred-moment statistic $\mathcal{M}^{(n)}_{\rho}$, to detect discrete phase-like transitions in the sampling dynamics of score-based diffusion models. By tracking forward diffusion and the conditional fluctuation tensors of two user-defined events, the authors reveal sharp mergers at specific steps, enabling early stopping, class-wise interval guidance, and improvements in zero-shot classification and style transfer without retraining. They derive a closed form for cross-fluctuations in variance-preserving SDEs, connect the approach to centered kernel alignment and classical coupling, and demonstrate practical gains across acceleration, conditional generation, rare-class synthesis, zero-shot classification, and zero-shot style transfer. The framework unifies discrete Markov-chain concepts with continuous stochastic dynamics, offering a scalable, theory-grounded toolkit for diagnosing and improving diffusion-model sampling. This bridges statistical-physics intuition with modern generative modeling, enabling efficient, targeted interventions that improve performance across multiple downstream tasks.

Abstract

We analyse how the sampling dynamics of distributions evolve in score-based diffusion models using cross-fluctuations, a centered-moment statistic from statistical physics. Specifically, we show that starting from an unbiased isotropic normal distribution, samples undergo sharp, discrete transitions, eventually forming distinct events of a desired distribution while progressively revealing finer structure. As this process is reversible, these transitions also occur in reverse, where intermediate states progressively merge, tracing a path back to the initial distribution. We demonstrate that these transitions can be detected as discontinuities in $n^{\text{th}}$-order cross-fluctuations. For variance-preserving SDEs, we derive a closed-form for these cross-fluctuations that is efficiently computable for the reverse trajectory. We find that detecting these transitions directly boosts sampling efficiency, accelerates class-conditional and rare-class generation, and improves two zero-shot tasks--image classification and style transfer--without expensive grid search or retraining. We also show that this viewpoint unifies classical coupling and mixing from finite Markov chains with continuous dynamics while extending to stochastic SDEs and non Markovian samplers. Our framework therefore bridges discrete Markov chain theory, phase analysis, and modern generative modeling.

Figures (22)

• Figure 1: The addition of noise causes distinct categories of data to "merge" through the forward diffusion process as statistical properties progressively converge to those of the standard normal distribution.
• Figure 2: Visualizing the merger cascade. This temporal hierarchy illustrates how distinct, mutually disjoint events merge as the diffusion process evolves. Time $t$ flows upward. At $t=0$, events are distinct leaves. As they diffuse, pairs whose fluctuation tensors become indistinguishable undergo a discrete merger event (black dot), and their branches merge. This cascade continues until all discriminating information has vanished into a single cluster.
• Figure 3: Fourth order generative diagram for CIFAR10.We show the emergence of classes using fourth-order correlations.
• Figure 4: Fourth order generative diagram for MNIST.We show the emergence of classes using fourth-order correlations.
• Figure 5: Conceptual visualisation of lattice transition dynamics.(a) The phase diagram illustrates how the number of merger events changes with the threshold $\epsilon$. Adopting a lattice perspective reveals a spectrum of distinct phases: an unrealizable thermodynamic phase at $\epsilon =0$ which admits a single speciation transition, an unrealizable region below the system's precision limit, a realizable plateau of maximum potential mergers, a staircase of sequential transitions where the system "splits" into fine-grained merger states, and a final saturated state. (b) The feasibility frontiers visualise the trade-off between merger time $t^*$ and threshold $\epsilon$. The relationship is analogous to an uncertainty principle, where higher-order fluctuations merge faster, shifting their feasibility frontiers to the left.

Theorems & Definitions (22)

• Example 1
• Theorem 2: Bochner's theorem for $\mathbb{R}^d$
• proof
• Remark 3: Existence of the characteristic function
• Theorem 4
• proof
• Theorem 5: Moment–TV inequality
• proof
• Remark 6
• Theorem 7: chen2022sampling