Differential equation and probability inspired graph neural networks for latent variable learning
Zhuangwei Shi
TL;DR
This paper tackles latent-variable learning by framing subspace learning on graphs through differential equations and probabilistic models. It develops PDE- and ODE-inspired graph neural networks (collectively termed POGNN) that connect continuous dynamics, diffusion processes, and conditional random fields to learn low-dimensional latent representations from high-dimensional observations. Key contributions include (i) connecting Neural ODEs with GNN dynamics, (ii) PDE-inspired channels that leverage heat diffusion, spectral kernels, and spatial attention, and (iii) a CRF-based energy perspective with first- and second-order Laplacian terms guiding GNN update rules. The framework yields interpretable dynamics and improved performance on standard benchmarks, illustrating the practical impact of integrating differential equations and probabilistic reasoning into graph-based latent-variable learning.
Abstract
Probabilistic theory and differential equation are powerful tools for the interpretability and guidance of the design of machine learning models, especially for illuminating the mathematical motivation of learning latent variable from observation. Subspace learning maps high-dimensional features on low-dimensional subspace to capture efficient representation. Graphs are widely applied for modeling latent variable learning problems, and graph neural networks implement deep learning architectures on graphs. Inspired by probabilistic theory and differential equations, this paper conducts notes and proposals about graph neural networks to solve subspace learning problems by variational inference and differential equation.