Multi-Output Robust and Conjugate Gaussian Processes

TL;DR

This work extends robust and conjugate Gaussian processes (RCGPs) to the multi-output setting, producing MO-RCGPs that jointly model correlated outputs with exact conjugate posteriors. By introducing a multivariate weight framework and a conditional centering approach across outputs, MO-RCGPs achieve provable robustness to outliers while preserving analytical tractability. The paper derives closed-form posterior and predictive distributions, proves robustness properties, and proposes a robust, scalable hyperparameter optimization via weighted LOO-CV. Empirical results on synthetic data, energy efficiency, cancer drug response, and financial data demonstrate competitive performance with substantially reduced sensitivity to outliers compared to standard MOGPs and competitive with $t$-MOGPs, often at a lower computational cost. These MO-RCGPs offer a practical, robust alternative for applications requiring reliable multi-output regression under misspecification and contamination.

Abstract

Multi-output Gaussian process (MOGP) regression allows modelling dependencies among multiple correlated response variables. Similarly to standard Gaussian processes, MOGPs are sensitive to model misspecification and outliers, which can distort predictions within individual outputs. This situation can be further exacerbated by multiple anomalous response variables whose errors propagate due to correlations between outputs. To handle this situation, we extend and generalise the robust and conjugate Gaussian process (RCGP) framework introduced by Altamirano et al. (2024). This results in the multi-output RCGP (MO-RCGP): a provably robust MOGP that is conjugate, and jointly captures correlations across outputs. We thoroughly evaluate our approach through applications in finance and cancer research.

Figures (6)

• Figure 1: Navitoclax Dose-Response with Outlier. We show the viability of two cancer cell lines exposed to varying doses of the drug Navitoclax (data available from the https://www.cancerrxgene.org/). The MOGP predictions are sensitive to the outlier (red dot), while the MO-RCGP predictions are robust.
• Figure 2: Synthetic Imputation with Focused Outliers. We generate $N=120$ data points from an ICM with parameters $B_{11} = 2$, $B_{22} = 1$, and $B_{12}=1.25$, contaminate $\epsilon_2 = 2.5\%$ of the observations in the second output and remove data in $y_1$ where $0.3<x<0.7$. The prior mean is $m_t(\mathbf{x}) = \frac{1}{N_t}\sum_{i=1}^{N_t} y_{i,t}$. The MOGP predictives are sensitive to outliers, and the errors propagate to the other output. In contrast, the MO-RCGP predictives remain robust across both outputs.
• Figure 3: Robustness in Mortgage-Backed Security Trades. We apply MOGP and MO-RCGP regression to $T=3$ MBS using an ICM with $m_t(\mathbf{x}) = \frac{1}{N}\sum_{i=1}^{N} y_{i,t}$. The outlier at 4 pm in the 5.5% MBS distorts the MOGP predictions for that output and propagates errors to the 6% MBS. In contrast, MO-RCGP is robust.
• Figure 4: Simulated Data with Focused Outliers. MO-RCGP ($w_{ \textup{MORCGP}}$) uses the weight function described in this paper, while the MO-RCGP ($w_{ \textup{RCGP}}$) use the original RCGP weight with $\gamma_t(\mathbf{x}, \mathbf{y}) = m_t(\mathbf{x}) = \frac{1}{N}\sum^N_{i=1}y_{i,t}$. Among all methods, only MO-RCGP ($w_{ \textup{MORCGP}}$) is robust to the contamination.
• Figure 5: Performance of MOGP and MO-RCGP as the number of outliers in the observed vector increases. We generate $N=80$ data points from an ICM across $T=10$ outputs. We then contaminate $\epsilon_s = 10\%$ of the observations with multivariate outliers for $s \in \{1, \dots, S\}$ with $S \leq T$ increasing incrementally. The prior mean is $m_t(\mathbf{x}) = \frac{1}{N_t}\sum_{i=1}^{N_t} y_{i,t}$. As the percentage of outliers grows, the performance of the MOGP drops substantially faster than that of the MO-RCGP.

Theorems & Definitions (16)

• Proposition 3.1
• Proposition 3.2
• Lemma B.1
• proof
• Lemma B.2
• proof
• proof
• Lemma B.3
• proof
• Lemma B.4