Partially Observable Gaussian Process Network and Doubly Stochastic Variational Inference

TL;DR

The paper tackles modeling real-world process networks where intermediate subprocess outputs are only partially observable. It introduces Partially Observable Gaussian Process Network (POGPN), a DAG-based GP framework that places latent functions in a common space and uses observation lenses to model indirect measurements, combining with doubly stochastic variational inference. The authors develop two training schemes (ancestor-wise and node-wise) and two objective losses (ELBO and PLL) to perform joint inference over the network, accommodating non-Gaussian likelihoods and multimodal observations. Across Jura, EEG, and synthetic benchmarks, POGPN delivers superior predictive performance and robust handling of stochastic parents and intermediate data, illustrating its practical potential for complex process networks.

Abstract

To reduce the curse of dimensionality for Gaussian processes (GP), they can be decomposed into a Gaussian Process Network (GPN) of coupled subprocesses with lower dimensionality. In some cases, intermediate observations are available within the GPN. However, intermediate observations are often indirect, noisy, and incomplete in most real-world systems. This work introduces the Partially Observable Gaussian Process Network (POGPN) to model real-world process networks. We model a joint distribution of latent functions of subprocesses and make inferences using observations from all subprocesses. POGPN incorporates observation lenses (observation likelihoods) into the well-established inference method of deep Gaussian processes. We also introduce two training methods for POPGN to make inferences on the whole network using node observations. The application to benchmark problems demonstrates how incorporating partial observations during training and inference can improve the predictive performance of the overall network, offering a promising outlook for its practical application.

Figures (8)

• Figure 1: Example process network where stochastic subprocesses are coupled by the latent state $\bm{f} ^{(\cdot)}$ which is partially observable as $\bm{y} ^{(\cdot)}$. Along with the input from the parent, a subprocess can also have adjustable input $\bm{x} ^{(\cdot)}.$
• Figure 2: Comparison of GP network and POGPN. Gray nodes represent observed outputs (likelihood), and white nodes represent latent outputs (GP).
• Figure 3: Training methods POGPN for a given structure. If $\mathcal{W}_{\text{obs}}=\{ \bm{y} ^{(4)}, \bm{y} ^{(5)}\}$, POGPN-AL includes hyperparameters for node $\bm{\gamma} ^{(\mathcal{W}_{\text{obs}})}=( \bm{\lambda} ^{(\mathcal{W}_{\text{obs}})}, \bm{\lambda} ^{(\text{Anc}(\mathcal{W}_{\text{obs}}))}, \bm{\theta} ^{(\mathcal{W}_{\text{obs}})})$ bounded by the blue dashed box. POGPN-NL trains hyperparameters, $\bm{\gamma} ^{(w)} = ( \bm{\lambda} ^{(w)}, \bm{\theta} ^{(w)}), \forall w\in\mathcal{W}_{\text{obs}}$, node-wise as bounded by red dashed boxes. Gray nodes represent observed output nodes (likelihood), and white nodes represent latent output nodes (GP).
• Figure 4: Structure of POGPN with root node location "Loc", softmax likelihoods for "Rock" and "Land", and multi-task Gaussian likelihood for minerals.
• Figure 5: Structure of POGPN for EEG dataset in Figure \ref{['fig:eeg_dag']}. Figure \ref{['fig:eeg_pred']} shows prediction results for sensor F2 using POGPN-AL (PLL) and GPN with a similar concept as described by Figure \ref{['fig:toy_process_gpn']}.