Bias mitigation in graph diffusion models

Abstract

Most existing graph diffusion models have significant bias problems. We observe that the forward diffusion's maximum perturbation distribution in most models deviates from the standard Gaussian distribution, while reverse sampling consistently starts from a standard Gaussian distribution, which results in a reverse-starting bias. Together with the inherent exposure bias of diffusion models, this results in degraded generation quality. This paper proposes a comprehensive approach to mitigate both biases. To mitigate reverse-starting bias, we employ a newly designed Langevin sampling algorithm to align with the forward maximum perturbation distribution, establishing a new reverse-starting point. To address the exposure bias, we introduce a score correction mechanism based on a newly defined score difference. Our approach, which requires no network modifications, is validated across multiple models, datasets, and tasks, achieving state-of-the-art results.Code is at https://github.com/kunzhan/spp

Figures (7)

• Figure 2: (a) and (b) The $\ell_2$ norm of the predictions from the two score networks at different time steps. (c) The generation results of two score networks with perturbations at different time steps.
• Figure 3: Score correction based on the score difference. At the reverse-sampling time $t$, the optimal score always points to ${{\bm X}}_0$. ${{\bm s}}_{{\bm\theta},t}(\cdot)$ points to $\gamma_t{{\bm X}}_0$ with some deviation (partially containing ${{\bm X}}_0$), while ${{\bm s}}_{{\bm\psi},t}(\cdot)$ points to $\gamma'_t{{\bm X}}_0$ with larger deviation (containing little ${{\bm X}}_0$). The difference between predicted and pseudo scores guides the predicted score towards the optimal score. We use $\lambda$ to control the angle and $\omega$ to adjust the magnitude of the corrected score. The final corrected score flexibly approaches the optimal score within the dashed box.
• Figure 4: Generation metric responses to perturbations at different time steps for two score networks.
• Figure : (a) Reverse-starting bias
• Figure : (a) Reverse-starting bias