Weighted-Sum of Gaussian Process Latent Variable Models

TL;DR

This work addresses signal separation when observations arise from a weighted sum of unknown, non-parametrically varying pure component signals. It introduces Weighted-Sum GPLVM (WS-GPLVM), a Bayesian non-parametric model that places GP priors on each pure component $f_c$, with latent factors $h_i$ capturing unobserved conditions and mixture weights $r_{i\cdot}$ constrained by $\sum_c r_{ic}=1$, using inducing points and variational inference to derive an ELBO for posterior estimation. Key contributions include the formulation of the WS-GPLVM and WS-GPLVM-ind, a tractable variational framework with an analytically computable ELBO, and a detailed training procedure to encourage balanced use of latent variables and weights; the model is demonstrated on spectroscopy and other domains with compelling results against baseline methods like ILMC, CLS-GP, and PLS. The approach enables flexible, non-linear variation modeling of pure component signals under limited labeled data and can be applied to spectroscopy and broader signal-processing problems where linear mixture assumptions are insufficient.

Abstract

This work develops a Bayesian non-parametric approach to signal separation where the signals may vary according to latent variables. Our key contribution is to augment Gaussian Process Latent Variable Models (GPLVMs) for the case where each data point comprises the weighted sum of a known number of pure component signals, observed across several input locations. Our framework allows arbitrary non-linear variations in the signals while being able to incorporate useful priors for the linear weights, such as summing-to-one. Our contributions are particularly relevant to spectroscopy, where changing conditions may cause the underlying pure component signals to vary from sample to sample. To demonstrate the applicability to both spectroscopy and other domains, we consider several applications: a near-infrared spectroscopy dataset with varying temperatures, a simulated dataset for identifying flow configuration through a pipe, and a dataset for determining the type of rock from its reflectance.

Figures (6)

• Figure 1: Bayesian diagram for Weighted-Sum of Gaussian Process latent variable models (WS-GPLVM) with unobserved mixture weights $r_{ic}$.
• Figure 2: Illustrative example of WS-GPLVM: by inputting a set of data $(a)$ of training data (top), with known mixture fractions, and test data (bottom), with unknown mixture fractions, WS-GPLVM produces variational posterior estimates of $(b)$ the fractional composition of the test data (top left), the latent variables of both the training and the test data (bottom left), and a Gaussian Process estimate, shown with a mean and $95\%$ confidence interval, for the pure component signals (right).
• Figure 3: (a) Shows the training and test data after SNV was applied. Each orange line represents a training data point and each blue line is a test data point. (b) Shows estimated mass fractions. The blue points show the true values, the black crosses show the mean of our method's variational posterior, the orange points are 100 samples drawn from the variational posterior. (c) and (d) show the latent variables colored by temperature (purple for $30^\circ C$, yellow for 70$^\circ C$) which we expected to cause variation in the pure signals, and water fraction (purple for no water, yellow for pure water), where the variation is induced by the normalizing procedure. (e) Mean estimated pure component signals for the training and test spectra, evaluated at the mean of the learned latent variable and colored by the temperature of the measurement (red for $30^\circ C$, yellow for 70$^\circ C$).
• Figure 4: Mean squared error (MSE) and negative log predictive density (NLPD) for 10 train-test splits of the a) near infra-red spectroscopy and b) oil flow datasets.
• Figure 5: Accuracy, log predictive probability (LPP) and area under receiver operator curve (ROC AUC) for remote sensing rock classification for a 10-fold cross-validation.