Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unifying Generation and Prediction on Graphs with Latent Graph Diffusion

Cai Zhou, Xiyuan Wang, Muhan Zhang

TL;DR

This paper first formulate prediction tasks including regression and classification into a generic (conditional) generation framework, which enables diffusion models to perform deterministic tasks with provable guarantees, and proposes Latent Graph Diffusion (LGD), a generative model that can generate node, edge, and graph-level features of all categories simultaneously.

Abstract

In this paper, we propose the first framework that enables solving graph learning tasks of all levels (node, edge and graph) and all types (generation, regression and classification) using one formulation. We first formulate prediction tasks including regression and classification into a generic (conditional) generation framework, which enables diffusion models to perform deterministic tasks with provable guarantees. We then propose Latent Graph Diffusion (LGD), a generative model that can generate node, edge, and graph-level features of all categories simultaneously. We achieve this goal by embedding the graph structures and features into a latent space leveraging a powerful encoder and decoder, then training a diffusion model in the latent space. LGD is also capable of conditional generation through a specifically designed cross-attention mechanism. Leveraging LGD and the ``all tasks as generation'' formulation, our framework is capable of solving graph tasks of various levels and types. We verify the effectiveness of our framework with extensive experiments, where our models achieve state-of-the-art or highly competitive results across a wide range of generation and regression tasks.

Unifying Generation and Prediction on Graphs with Latent Graph Diffusion

TL;DR

This paper first formulate prediction tasks including regression and classification into a generic (conditional) generation framework, which enables diffusion models to perform deterministic tasks with provable guarantees, and proposes Latent Graph Diffusion (LGD), a generative model that can generate node, edge, and graph-level features of all categories simultaneously.

Abstract

In this paper, we propose the first framework that enables solving graph learning tasks of all levels (node, edge and graph) and all types (generation, regression and classification) using one formulation. We first formulate prediction tasks including regression and classification into a generic (conditional) generation framework, which enables diffusion models to perform deterministic tasks with provable guarantees. We then propose Latent Graph Diffusion (LGD), a generative model that can generate node, edge, and graph-level features of all categories simultaneously. We achieve this goal by embedding the graph structures and features into a latent space leveraging a powerful encoder and decoder, then training a diffusion model in the latent space. LGD is also capable of conditional generation through a specifically designed cross-attention mechanism. Leveraging LGD and the ``all tasks as generation'' formulation, our framework is capable of solving graph tasks of various levels and types. We verify the effectiveness of our framework with extensive experiments, where our models achieve state-of-the-art or highly competitive results across a wide range of generation and regression tasks.
Paper Structure (66 sections, 4 theorems, 40 equations, 1 figure, 11 tables)

This paper contains 66 sections, 4 theorems, 40 equations, 1 figure, 11 tables.

Table of Contents

  1. Introduction
  2. Our work.
  3. Related Work
  4. Diffusion models.
  5. Diffusion models for graphs.
  6. Regression and Classification as Conditional Generation
  7. Latent Graph Diffusion
  8. Notations.
  9. Overview
  10. Autoencoding
  11. Architecture.
  12. Training.
  13. Diffusion model
  14. Architecture.
  15. Training and sampling.
  16. ...and 51 more sections

Key Result

Theorem 3.1

Suppose assumption_error, assumption_lipschitz, assumption_score_estimation hold, and the step size $h:=T/N$ satisfies $h\preceq 1/L$ where $L\geq 1$. Then the mean absolute error (MAE) of the conditional latent diffusion model in the regression task is bounded by where $q_z$ is the ground truth distribution of ${\bm{z}}:=w^t(\tau({\bm{x}}, y)-\mathcal{E}({\bm{x}}))$.

Figures (1)

  • Figure 1: Illustration of the Latent Graph Diffusion framework, which is capable of performing both generation and prediction.

Theorems & Definitions (6)

  • Theorem 3.1
  • Theorem B.4
  • proof
  • Lemma B.5
  • Corollary B.6
  • proof