Estimation of log-Gaussian gamma processes with iterated posterior linearization and Hamiltonian Monte Carlo

TL;DR

This work addresses posterior inference for the log-Gaussian gamma process, where measurements $y_k$ arise from a Gamma distribution with shape and rate parameters given by exponentials of latent GP functions $\boldsymbol{\alpha}$ and $\boldsymbol{\beta}$. It introduces two IPL-based schemes that fuse iterated posterior linearization with Hamiltonian Monte Carlo to efficiently sample the posterior $\pi(\boldsymbol{\alpha},\boldsymbol{\beta},\mu_\alpha,\theta_\alpha,\mu_\beta,\theta_\beta\mid \boldsymbol{y})$: (i) an approximate PL+HMC method and (ii) a tempered PL method that drives a sequence of tempered posteriors to the true target. The results on synthetic LGG data, a multiscale stiffness model, and argentopyrite Raman spectra demonstrate substantial speedups (around 17–26x) with comparable accuracy to full HMC, while tempering delivers superior mixing. The approaches are readily applicable to similar latent Gaussian process models, including log-Gaussian Cox processes, and advance efficient Bayesian inference for non-Gaussian measurement models in spectroscopy and materials science, with accompanying open-source software.

Abstract

Stochastic processes are a flexible and widely used family of models for statistical modeling. While stochastic processes offer attractive properties such as inclusion of uncertainty properties, their inference is typically intractable, with the notable exception of Gaussian processes. Inference of models with non-Gaussian errors typically involves estimation of a high-dimensional latent variable. We propose two methods that use iterated posterior linearization followed by Hamiltonian Monte Carlo to sample the posterior distributions of such latent models with a particular focus on log-Gaussian gamma processes. The proposed methods are validated with two synthetic datasets generated from the log-Gaussian gamma process and a multiscale biocomposite stiffness model. In addition, we apply the methodology to an experimental Raman spectrum of argentopyrite.

Figures (17)

• Figure 1: The log-Gaussian gamma process. Measurement data $\boldsymbol{y}$ is modeled as gamma-distributed random variables. The log-shape and log-rate parameters $\boldsymbol{\alpha}$ and $\boldsymbol{\beta}$ of the gamma distribution are modeled as Gaussian processes with parameters $\mu_\alpha$, $\boldsymbol{\theta}_\alpha$, $\mu_\beta$, and $\boldsymbol{\theta}_\beta$. $\mathbb{E}\left[ \pi( \boldsymbol{y} \mid \boldsymbol{\alpha}, \boldsymbol{\beta} ) \right]$ and $\textrm{Var}_{90\%}\left[ \pi( \boldsymbol{y} \mid \boldsymbol{\alpha}, \boldsymbol{\beta} ) \right]$ are the mean and 90% confidence interval of the log-Gaussian gamma process.
• Figure 2: An example sequence of iterated posterior linearization for the log-shape process $\boldsymbol{\alpha}$ corresponding to the case shown in Figure \ref{['im:lggpDiagram']} with $T = 5$ iterations. Starting from the prior distribution at $t = 0$, the mean and covariance estimates are iteratively updated and refined until convergence. The 90% marginal credible interval defined by the covariance $\boldsymbol{P}_{\alpha}^{(t)}$ is denoted by $\boldsymbol{P}_{\alpha, 90\%}^{(t)}$.
• Figure 3: Synthetic dataset. In (a), data in blue, the ground truth expectation, and 90% confidence interval of the data-generating function in black and gray, respectively, and $\boldsymbol{\alpha}_\textrm{GT}$ and $\boldsymbol{\beta}_\textrm{GT}$ denote the ground truth log-shape and log-rate processes. The mean estimate $\mathbb{E}\left[ \pi( \boldsymbol{y} \mid \boldsymbol{\alpha}, \boldsymbol{\beta} ) \right]$ and the corresponding 90% credible interval $\textrm{Var}_{90\%}\left[ \pi( \boldsymbol{y} \mid \boldsymbol{\alpha}, \boldsymbol{\beta} ) \right]$ for the data-generating function obtained with (b) HMC, (c) short chain HMC, (d) iterated posterior linearization followed by HMC sampling, and (e) iterated posterior linearization and tempered short chain HMC sampling.
• Figure 4: Synthetic dataset. The mean estimates $\mathbb{E}\left[ \pi( \boldsymbol{\alpha} \mid \boldsymbol{y} ) \right]$ and $\mathbb{E}\left[ \pi( \boldsymbol{\beta} \mid \boldsymbol{y} ) \right]$ with the corresponding 90% credible intervals $\textrm{Var}_{90\%}\left[ \pi( \boldsymbol{\alpha} \mid \boldsymbol{y} ) \right]$ and $\textrm{Var}_{90\%}\left[ \pi( \boldsymbol{\beta} \mid \boldsymbol{y} ) \right]$ for the log-shape and log-rate processes obtained with (a) HMC sampling and (b) iterated posterior linearization followed by HMC sampling. The ground truth log-shape and log-rate processes are denoted by $\boldsymbol{\alpha}_\textrm{GT}$ and $\boldsymbol{\beta}_\textrm{GT}$.
• Figure 5: Synthetic dataset. The mean estimates $\mathbb{E}\left[ \pi( \boldsymbol{\alpha} \mid \boldsymbol{y} ) \right]$ and $\mathbb{E}\left[ \pi( \boldsymbol{\beta} \mid \boldsymbol{y} ) \right]$ with the corresponding 90% credible intervals $\textrm{Var}_{90\%}\left[ \pi( \boldsymbol{\alpha} \mid \boldsymbol{y} ) \right]$ and $\textrm{Var}_{90\%}\left[ \pi( \boldsymbol{\beta} \mid \boldsymbol{y} ) \right]$ for the log-shape and log-rate processes obtained with (a) HMC sampling, (b) short chain HMC sampling, and (c) iterated posterior linearization and tempered short chain HMC. The ground truth log-shape and log-rate processes are denoted by $\boldsymbol{\alpha}_\textrm{GT}$ and $\boldsymbol{\beta}_\textrm{GT}$.