Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Gaussian Processes for Functional Maps

Matthew Lowery, Zhitong Xu, Da Long, Keyan Chen, Daniel S. Johnson, Yang Bai, Varun Shankar, Shandian Zhe

TL;DR

The paper introduces Deep Gaussian Processes for Functional Maps (DGPFM), a hierarchical GP-based approach to function-on-function regression that learns nonlinear mappings with principled uncertainty. By representing the mapping as a sequence of linear and nonlinear GP transforms and exploiting a discrete integral transform that cancels under GP interpolation, DGPFM remains scalable and flexible to irregular sampling. Variational inference with inducing points and whitening enables tractable training and uncertainty quantification, achieving superior predictive accuracy and well-calibrated uncertainty on PDE benchmarks and real-world function-to-function tasks. The work demonstrates that DGPFM can outperform functional linear models and neural operator baselines, particularly in challenging irregular-sampling settings, with practical implications for spatiotemporal forecasting and climate modeling.

Abstract

Learning mappings between functional spaces, also known as function-on-function regression, plays a crucial role in functional data analysis and has broad applications, e.g. spatiotemporal forecasting, curve prediction, and climate modeling. Existing approaches, such as functional linear models and neural operators, either fall short of capturing complex nonlinearities or lack reliable uncertainty quantification under noisy, sparse, and irregularly sampled data. To address these issues, we propose Deep Gaussian Processes for Functional Maps (DGPFM). Our method designs a sequence of GP-based linear and nonlinear transformations, leveraging integral transforms of kernels, GP interpolation, and nonlinear activations sampled from GPs. A key insight simplifies implementation: under fixed locations, discrete approximations of kernel integral transforms collapse into direct functional integral transforms, enabling flexible incorporation of various integral transform designs. To achieve scalable probabilistic inference, we use inducing points and whitening transformations to develop a variational learning algorithm. Empirical results on real-world and PDE benchmark datasets demonstrate that the advantage of DGPFM in both predictive performance and uncertainty calibration.

Deep Gaussian Processes for Functional Maps

TL;DR

The paper introduces Deep Gaussian Processes for Functional Maps (DGPFM), a hierarchical GP-based approach to function-on-function regression that learns nonlinear mappings with principled uncertainty. By representing the mapping as a sequence of linear and nonlinear GP transforms and exploiting a discrete integral transform that cancels under GP interpolation, DGPFM remains scalable and flexible to irregular sampling. Variational inference with inducing points and whitening enables tractable training and uncertainty quantification, achieving superior predictive accuracy and well-calibrated uncertainty on PDE benchmarks and real-world function-to-function tasks. The work demonstrates that DGPFM can outperform functional linear models and neural operator baselines, particularly in challenging irregular-sampling settings, with practical implications for spatiotemporal forecasting and climate modeling.

Abstract

Learning mappings between functional spaces, also known as function-on-function regression, plays a crucial role in functional data analysis and has broad applications, e.g. spatiotemporal forecasting, curve prediction, and climate modeling. Existing approaches, such as functional linear models and neural operators, either fall short of capturing complex nonlinearities or lack reliable uncertainty quantification under noisy, sparse, and irregularly sampled data. To address these issues, we propose Deep Gaussian Processes for Functional Maps (DGPFM). Our method designs a sequence of GP-based linear and nonlinear transformations, leveraging integral transforms of kernels, GP interpolation, and nonlinear activations sampled from GPs. A key insight simplifies implementation: under fixed locations, discrete approximations of kernel integral transforms collapse into direct functional integral transforms, enabling flexible incorporation of various integral transform designs. To achieve scalable probabilistic inference, we use inducing points and whitening transformations to develop a variational learning algorithm. Empirical results on real-world and PDE benchmark datasets demonstrate that the advantage of DGPFM in both predictive performance and uncertainty calibration.
Paper Structure (24 sections, 29 equations, 5 figures, 8 tables)

This paper contains 24 sections, 29 equations, 5 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Model
  4. GP-based Linear and Nonlinear Transformation
  5. Model Framework
  6. Algorithm
  7. Discrete Approximation of Integral Transforms
  8. Stochastic Variational Learning
  9. Related Work
  10. Experiment
  11. Conclusion
  12. GP Covariance for Integral Transformation
  13. Dataset Details
  14. 1D Burgers's equation
  15. 2D Darcy Flow
  16. ...and 9 more sections

Figures (5)

  • Figure 1: Prediction examples of DGPFM on Beijing-Air dataset. The shaded regions indicate one predictive standard deviation. The top row shows the prediction of DGPFM-FT and the bottom row DGPFM-QR.
  • Figure 2: Prediction examples of DGPFM on Quasar dataset. The shaded regions indicate one predictive standard deviation. The top row shows the prediction of DGPFM-FT and the bottom row DGPFM-QR.
  • Figure 3: Prediction examples of DGPFM-QR on 2D Darcy, $\sigma$ denotes the predictive standard deviation (STD). The last two columns show the point-wise predictive std normalized by the ground-truth.
  • Figure 4: Prediction examples of DGPFM-FT on 1D Burgers.
  • Figure 5: The Singular values of learned weight function values (matrices) versus randomly initialized matrices for running DGPFM-QR on 2D Darcy.