Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models

Marcos Villagra, Bidhan Roy, Raihan Seraj, Zhiying Jiang

TL;DR

This paper decouples numerical stability from generation quality in Decentralized Diffusion Models, showing that expert-data alignment -- routing inputs to experts trained on similar data -- is the primary determinant of sample quality. Through analyses on Paris-style multi-expert DDMs and MNIST-based validation, it demonstrates that sparse Top-2 routing yields superior FID by preserving alignment, while full ensemble achieves smoother dynamics yet degrades perceptual quality due to expert disagreement. A trajectory-sensitivity framework reveals a stability–dissociation: lower trajectory sensitivity does not guarantee better samples, and within-strategy diagnostics like $\widehat{L}_{\text{eff}}^{(h)}$ alone cannot predict cross-strategy quality. Practically, the work suggests prioritizing data-alignment-aware routing for deployment and points to potential computational gains from sparse routing. The findings advance our understanding of how distributed experts can coherently generate data and inform routing design for scalable diffusion systems.

Abstract

Decentralized Diffusion Models (DDMs) route denoising through experts trained independently on disjoint data clusters, which can strongly disagree in their predictions. What governs the quality of generations in such systems? We present the first ever systematic investigation of this question. A priori, the expectation is that minimizing denoising trajectory sensitivity -- minimizing how perturbations amplify during sampling -- should govern generation quality. We demonstrate this hypothesis is incorrect: a stability-quality dissociation. Full ensemble routing, which combines all expert predictions at each step, achieves the most stable sampling dynamics and best numerical convergence while producing the worst generation quality (FID 47.9 vs. 22.6 for sparse Top-2 routing). Instead, we identify expert-data alignment as the governing principle: generation quality depends on routing inputs to experts whose training distribution covers the current denoising state. Across two distinct DDM systems, we validate expert-data alignment using (i) data-cluster distance analysis, confirming sparse routing selects experts with data clusters closest to the current denoising state, and (ii) per-expert analysis, showing selected experts produce more accurate predictions than non-selected ones, and (iii) expert disagreement analysis, showing quality degrades when experts disagree. For DDM deployment, our findings establish that routing should prioritize expert-data alignment over numerical stability metrics.

Expert-Data Alignment Governs Generation Quality in Decentralized Diffusion Models

TL;DR

This paper decouples numerical stability from generation quality in Decentralized Diffusion Models, showing that expert-data alignment -- routing inputs to experts trained on similar data -- is the primary determinant of sample quality. Through analyses on Paris-style multi-expert DDMs and MNIST-based validation, it demonstrates that sparse Top-2 routing yields superior FID by preserving alignment, while full ensemble achieves smoother dynamics yet degrades perceptual quality due to expert disagreement. A trajectory-sensitivity framework reveals a stability–dissociation: lower trajectory sensitivity does not guarantee better samples, and within-strategy diagnostics like alone cannot predict cross-strategy quality. Practically, the work suggests prioritizing data-alignment-aware routing for deployment and points to potential computational gains from sparse routing. The findings advance our understanding of how distributed experts can coherently generate data and inform routing design for scalable diffusion systems.

Abstract

Decentralized Diffusion Models (DDMs) route denoising through experts trained independently on disjoint data clusters, which can strongly disagree in their predictions. What governs the quality of generations in such systems? We present the first ever systematic investigation of this question. A priori, the expectation is that minimizing denoising trajectory sensitivity -- minimizing how perturbations amplify during sampling -- should govern generation quality. We demonstrate this hypothesis is incorrect: a stability-quality dissociation. Full ensemble routing, which combines all expert predictions at each step, achieves the most stable sampling dynamics and best numerical convergence while producing the worst generation quality (FID 47.9 vs. 22.6 for sparse Top-2 routing). Instead, we identify expert-data alignment as the governing principle: generation quality depends on routing inputs to experts whose training distribution covers the current denoising state. Across two distinct DDM systems, we validate expert-data alignment using (i) data-cluster distance analysis, confirming sparse routing selects experts with data clusters closest to the current denoising state, and (ii) per-expert analysis, showing selected experts produce more accurate predictions than non-selected ones, and (iii) expert disagreement analysis, showing quality degrades when experts disagree. For DDM deployment, our findings establish that routing should prioritize expert-data alignment over numerical stability metrics.
Paper Structure (54 sections, 1 theorem, 11 equations, 4 figures, 15 tables)

This paper contains 54 sections, 1 theorem, 11 equations, 4 figures, 15 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Diffusion as ODE/SDE and numerical stability.
  4. Local Lipschitz analysis.
  5. Decentralized diffusion.
  6. DDM vs. traditional Mixture-of-Experts.
  7. Data-aware routing and expert specialization.
  8. Background
  9. Notation
  10. Diffusion sampling
  11. Decentralized Diffusion
  12. The Stability--Quality Dissociation
  13. Expert-Data Alignment
  14. The Expert-Data Alignment Hypothesis
  15. Cluster Distance Analysis
  16. ...and 39 more sections

Key Result

Proposition B.1

Let $v$ be a velocity field and $q$ a noise distribution. Suppose that for $x_1 \sim q$, the effective Lipschitz constant $L_{\emph{eff}}(x_1)$ (Definition def:trajectory-local-sensitivity) satisfies $P(L_{\emph{eff}}(x_1) < \infty) = 1$. Let $\tilde{x}^{(h)}_{t_N}$ denote the numerical solution at

Figures (4)

  • Figure 1: Higher expert disagreement degrades perceptual quality in full ensemble routing. Samples binned by trajectory-integrated disagreement (Q1=lowest, Q4=highest). Higher disagreement quartiles show greater perceptual distance (LPIPS) from the Top-2 reference, explaining why full ensemble underperforms sparse routing.
  • Figure 2: Cumulative IQR of the Jacobian spectral norm $\|\nabla_x v(x_t, t)\|$ as a measure of variability across sampling trajectories. The gap between Top-2 and other strategies widens as denoising progresses. Mid-trajectory timesteps ($t \in [0.1, 0.9]$); $n{=}20$ samples per strategy.
  • Figure 3: Correlation between trajectory sensitivity and step-refinement disagreement from Experiment 3 ($n{=}1000$ samples per routing strategy). (a) Spearman correlation $\rho(\widehat{L}_{\text{eff}}^{(h)}, \Delta_{\text{refine}})$ is weak across all strategies ($\rho < 0.08$), indicating that $L_{\text{eff}}$ is not a tight predictor of discretization error. (b) Mean step-refinement disagreement $\Delta_{\text{refine}}$ shows clear ordering: Full ensemble achieves the lowest discretization error (0.020), followed by Top-2 (0.051) and Top-1 (0.075).
  • Figure 4: Temporal profile of the two Jacobian terms under Top-2 and full ensembling. Left: Expert term $\|\sum_k w_k \nabla_x v_k\|$ (linear scale). Right: Router term $\|\sum_k v_k \nabla_x w_k\|$ (log scale). The router term dominates by 2--4 orders of magnitude, but both routing strategies show similar router contributions.

Theorems & Definitions (8)

  • Definition 3.1: DDM sampling convergence
  • Definition 6.1: Trajectory-local sensitivity
  • Definition 6.2
  • Remark 6.3: Circularity of the diagnostic
  • Definition 6.4: Sampler sensitivity
  • Proposition B.1: Conditional convergence
  • proof
  • Remark B.2