Adaptive posterior distributions for uncertainty analysis of covariance matrices in Bayesian inversion problems for multioutput signals

TL;DR

An adaptive importance sampling (AIS) scheme for multivariate Bayesian inversion problems, which is based in two main ideas: the variables of interest are split in two blocks and the inference takes advantage of known analytical optimization formulas.

Abstract

In this paper we address the problem of performing Bayesian inference for the parameters of a nonlinear multi-output model and the covariance matrix of the different output signals. We propose an adaptive importance sampling (AIS) scheme for multivariate Bayesian inversion problems, which is based in two main ideas: the variables of interest are split in two blocks and the inference takes advantage of known analytical optimization formulas. We estimate both the unknown parameters of the multivariate non-linear model and the covariance matrix of the noise. In the first part of the proposed inference scheme, a novel AIS technique called adaptive target adaptive importance sampling (ATAIS) is designed, which alternates iteratively between an IS technique over the parameters of the non-linear model and a frequentist approach for the covariance matrix of the noise. In the second part of the proposed inference scheme, a prior density over the covariance matrix is considered and the cloud of samples obtained by ATAIS are recycled and re-weighted to obtain a complete Bayesian study over the model parameters and covariance matrix. ATAIS is the main contribution of the work. Additionally, the inverted layered importance sampling (ILIS) is presented as a possible compelling algorithm (but based on a conceptually simpler idea). Different numerical examples show the benefits of the proposed approaches

Figures (12)

• Figure 1: Graphical representation of the considered multioutput model with $K=3$ output signals and $R=4$ time instants for each signal. One can suppose that ${\bm \Sigma}$ represents a $3\times 3$ covariance matrix of three possible nodes in a graph.
• Figure 2: Location Example. MAE in the estimation of ${\bm \theta}_{\texttt{MAP}}$ (using ${\bm \theta}_{\texttt{true}}$ as the groundtruth), with different number of particles by ATAIS versus (a)$N$ with fixed $T$ and with (b)$T$ with fixed $N$.
• Figure 3: Location example. Convergence of the components in $\widehat{{\bm \theta}}_{\texttt{MAP}}^{(t)}$ and $\widehat{{\bm \Sigma}}_{\texttt{ML}}^{(t)}$ in one run of ATAIS algorithm.
• Figure 4: Location example: histogram of the components of ${\bm \Sigma}$, denoted as $[{\bm \Sigma}]_{i,j}=\bm{\Sigma}_{i,j}$, of the covariance matrices after resampling according to the weights $\bar{\lambda}_j$, for the location example.
• Figure 5: Location example: Estimations of $\bm{\theta}_{\texttt{MAP}}$ with different schemes. the green squares represent the estimation with ATAIS; the red circles depict for the estimations of ILIS with a covariance matrix generated from a Wishart distribution. The blue diamonds represent the estimations of the single MH addressing a conditional posterior as invariant density, where ${\bm \Sigma}$ is fixed to the value of estimation $\bm{\Sigma}_{\texttt{ML}}$ obtained by ATAIS. The black cross shows ${\bm \theta}_{\texttt{true}}$.