Spectral Convolutional Conditional Neural Processes

TL;DR

This paper introduces the Spectral Convolutional Conditional Neural Process (SConvCNP), a spectral, Fourier-domain extension of ConvCNPs that enables global receptive fields for probabilistic function prediction. By parameterizing convolution kernels in the frequency domain and leveraging FFTs, SConvCNP achieves scalable, translation-aware functional embeddings within the neural process framework, addressing limitations of local kernels. Across synthetic 1-D tasks, predator–prey dynamics, traffic flow, and image completion, the method shows competitive or superior performance relative to state-of-the-art NP baselines, while offering favorable computational efficiency. The work bridges neural processes with neural operator learning, highlighting the value of spectral methods for uncertainty-aware function-space inference and suggesting several practical directions for future enhancements like positional encodings and patch-based discretization.

Abstract

Neural processes (NPs) are probabilistic meta-learning models that map sets of observations to posterior predictive distributions, enabling inference at arbitrary domain points. Their capacity to handle variable-sized collections of unstructured observations, combined with simple maximum-likelihood training and uncertainty-aware predictions, makes them well-suited for modeling data over continuous domains. Since their introduction, several variants have been proposed. Early approaches typically represented observed data using finite-dimensional summary embeddings obtained through aggregation schemes such as mean pooling. However, this strategy fundamentally mismatches the infinite-dimensional nature of the generative processes that NPs aim to capture. Convolutional conditional neural processes (ConvCNPs) address this limitation by constructing infinite-dimensional functional embeddings processed through convolutional neural networks (CNNs) to enforce translation equivariance. Yet CNNs with local spatial kernels struggle to capture long-range dependencies without resorting to large kernels, which impose significant computational costs. To overcome this limitation, we propose the Spectral ConvCNP (SConvCNP), which performs global convolution in the frequency domain. Inspired by Fourier neural operators (FNOs) for learning solution operators of partial differential equations (PDEs), our approach directly parameterizes convolution kernels in the frequency domain, leveraging the relatively compact yet global Fourier representation of many natural signals. We validate the effectiveness of SConvCNP on both synthetic and real-world datasets, demonstrating how ideas from operator learning can advance the capabilities of NPs.

Figures (4)

• Figure 1: Example predictions for synthetic data. For each model, the blue curve shows its predictive mean and the shaded region indicates $\pm 2$ standard deviations under the corresponding Gaussian predictive distribution. In the first two rows, where the data are generated from a Gaussian process, the true joint density is shown in purple: the dash-dotted line marks the true mean, and the shaded band spans $\pm 2$ standard deviations around that mean. Red points denote query samples drawn from the ground-truth process, and black points indicate the context observations.
• Figure 2: Illustrative model predictions based on simulated trajectories from Lotka--Volterra predator--prey dynamics. Black points represent the context observations. For each model, the blue curve depicts the predictive mean, and the shaded region marks the $\pm 2$-standard-deviation band of the associated Gaussian predictive distribution. Ground-truth traffic measurements, shown in red, are included as a reference.
• Figure 3: Illustrative model predictions on the California traffic-flow dataset. Black points represent the context observations. For each model, the blue curve depicts the predictive mean, and the shaded region marks the $\pm 2$-standard-deviation band of the associated Gaussian predictive distribution. Ground-truth traffic measurements, shown in red, are included as a reference.
• Figure 4: Illustrative model outputs for image-completion tasks using samples from the DTD dataset. Grey pixels indicate query regions, and the remaining pixels serve as context observations. For each model, query pixels are filled with the mean of the predictive distribution.