Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generative Myopia: Why Diffusion Models Fail at Structure

Milad Siami

TL;DR

This work identifies Generative Myopia as the failure of likelihood-based graph diffusion to preserve rare but spectrally critical edges with high $R_eff$ under ELBO optimization. It proves a Failure Theorem showing gradient starvation prevents learning rare bridges and introduces Spectrally-Weighted Diffusion with a spectral prior that reweights the ELBO via a simple objective $L_RW$ to emphasize high-$R_eff$ edges, while keeping inference cost unchanged. The approach is validated through four controlled experiments (Barbell, Asymmetric Chain, Visible Bridge, and Optimization Dynamics) demonstrating that standard diffusion can miss critical connections, whereas the weighted method achieves near-perfect connectivity and matches a spectral oracle, including 100% connectivity on adversarial benchmarks. The results offer a principled path to fuse spectral graph theory with generative diffusion, enabling robust structure-aware graph synthesis in applications requiring global connectivity and resilience.

Abstract

Graph Diffusion Models (GDMs) optimize for statistical likelihood, implicitly acting as \textbf{frequency filters} that favor abundant substructures over spectrally critical ones. We term this phenomenon \textbf{Generative Myopia}. In combinatorial tasks like graph sparsification, this leads to the catastrophic removal of ``rare bridges,'' edges that are structurally mandatory ($R_{\text{eff}} \approx 1$) but statistically scarce. We prove theoretically and empirically that this failure is driven by \textbf{Gradient Starvation}: the optimization landscape itself suppresses rare structural signals, rendering them unlearnable regardless of model capacity. To resolve this, we introduce \textbf{Spectrally-Weighted Diffusion}, which re-aligns the variational objective using Effective Resistance. We demonstrate that spectral priors can be amortized into the training phase with zero inference overhead. Our method eliminates myopia, matching the performance of an optimal Spectral Oracle and achieving \textbf{100\% connectivity} on adversarial benchmarks where standard diffusion fails completely (0\%).

Generative Myopia: Why Diffusion Models Fail at Structure

TL;DR

This work identifies Generative Myopia as the failure of likelihood-based graph diffusion to preserve rare but spectrally critical edges with high under ELBO optimization. It proves a Failure Theorem showing gradient starvation prevents learning rare bridges and introduces Spectrally-Weighted Diffusion with a spectral prior that reweights the ELBO via a simple objective to emphasize high- edges, while keeping inference cost unchanged. The approach is validated through four controlled experiments (Barbell, Asymmetric Chain, Visible Bridge, and Optimization Dynamics) demonstrating that standard diffusion can miss critical connections, whereas the weighted method achieves near-perfect connectivity and matches a spectral oracle, including 100% connectivity on adversarial benchmarks. The results offer a principled path to fuse spectral graph theory with generative diffusion, enabling robust structure-aware graph synthesis in applications requiring global connectivity and resilience.

Abstract

Graph Diffusion Models (GDMs) optimize for statistical likelihood, implicitly acting as \textbf{frequency filters} that favor abundant substructures over spectrally critical ones. We term this phenomenon \textbf{Generative Myopia}. In combinatorial tasks like graph sparsification, this leads to the catastrophic removal of ``rare bridges,'' edges that are structurally mandatory () but statistically scarce. We prove theoretically and empirically that this failure is driven by \textbf{Gradient Starvation}: the optimization landscape itself suppresses rare structural signals, rendering them unlearnable regardless of model capacity. To resolve this, we introduce \textbf{Spectrally-Weighted Diffusion}, which re-aligns the variational objective using Effective Resistance. We demonstrate that spectral priors can be amortized into the training phase with zero inference overhead. Our method eliminates myopia, matching the performance of an optimal Spectral Oracle and achieving \textbf{100\% connectivity} on adversarial benchmarks where standard diffusion fails completely (0\%).

Paper Structure

This paper contains 26 sections, 2 theorems, 3 equations, 5 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contribution
  3. Related Work
  4. Generative Models on Graphs
  5. Spectral and Structural Limitations of Diffusion
  6. Spectral Graph Theory & Network Control
  7. Spectral Bias in Deep Learning
  8. Theoretical Analysis: The Generative Myopia Conflict
  9. Preliminaries
  10. The Failure Theorem
  11. Method: Spectrally-Weighted Diffusion
  12. Algorithm and Implementation
  13. Complexity Analysis
  14. Training (Amortized)
  15. Inference (Zero Cost)
  16. ...and 11 more sections

Key Result

Lemma 3.1

Consider a discrete diffusion model minimizing the variational lower bound $\mathcal{L}_{\text{ELBO}}$. Assuming a factored posterior $q(\mathbf{A}_{t-1}|\mathbf{A}_t, \mathbf{A}_0)$ (independent noise), the optimal reverse transition parameter $\theta^*$ for an edge $e_{ij}$ satisfies:

Figures (5)

  • Figure 1: The Mechanism of Generative Myopia. (1) The Ground Truth contains dense local clusters (Black) and a sparse global bridge (Red). (2) The Forward Process corrupts all edges equally. (3a) Standard Diffusion discards the bridge. (3b) Spectrally-Weighted Diffusion recovers it.
  • Figure 1: Experiment I: Barbell Graph.(A) Ground truth topology; the yellow bridge is the spectral bottleneck. (B) The Generative Gap: The bridge falls into the high-resistance/low-frequency "blind spot." (C) Results: Standard Diffusion (Red) fails. Weighted Diffusion (Green) recovers the topology.
  • Figure 2: Experiment II: Asymmetric Chain SBM Results.(A) Ground truth topology with variable cluster sizes. (B) The Generative Gap: Bridges (Red Dots) have high resistance but low frequency. (C) Performance: Random sampling (Gray) has decent success ($53\%$) due to feasible density. Standard Diffusion (Red) fails completely ($0\%$). Weighted Diffusion (Green) achieves perfect structural recovery ($100\%$), matching the Oracle.
  • Figure 3: The Cure for Myopia. We artificially increase the bridge frequency by thickening it ($k$ edges). Standard Diffusion (Red) exhibits a Phase Transition: it fails in the shaded "Zone of Myopia" ($k < 3$) where frequency is low, but succeeds once the bridge becomes statistically frequent ($k \ge 4$). Weighted Diffusion (Green) remains robust across all regimes, proving it relies on structural importance ($R_{\text{eff}}$) rather than statistical abundance.
  • Figure 4: Optimization Dynamics. We simulate the training of a neural network on a rare bridge ($P_{\text{freq}}=0.05$). Standard Diffusion (Red) suffers from Gradient Starvation, collapsing to the marginal frequency. Weighted Diffusion (Green) uses spectral weights to amplify the rare signal, driving the predicted probability to structural certainty ($p \approx 1.0$).

Theorems & Definitions (4)

  • Lemma 3.1: Convergence to Marginal Frequency
  • Proof 1
  • Theorem 3.2: Orthogonality of Likelihood and Connectivity
  • Proof 2