Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constructing structured tensor priors for Bayesian inverse problems

Kim Batselier

TL;DR

This paper develops a complete Gaussian prior framework for Bayesian inverse problems that enforces structured tensor solutions through (A,b)-constrained representations. It provides exact mean and covariance characterizations, including explicit constructions for permutation- invariant and Hankel-related structures, and introduces efficient sampling and kernel-based techniques to incorporate these priors in large-scale problems. The approach is demonstrated on Hankel matrix completion and MNIST classifier learning, with practical implementations via reactive Julia Pluto notebooks. The work opens a broad class of structured tensor priors and lays the groundwork for future scalable, kernelized Bayesian inference in high-dimensional tensor settings.

Abstract

Specifying a prior distribution is an essential part of solving Bayesian inverse problems. The prior encodes a belief on the nature of the solution and this regularizes the problem. In this article we completely characterize a Gaussian prior that encodes the belief that the solution is a structured tensor. We first define the notion of (A,b)-constrained tensors and show that they describe a large variety of different structures such as Hankel, circulant, triangular, symmetric, and so on. Then we completely characterize the Gaussian probability distribution of such tensors by specifying its mean vector and covariance matrix. Furthermore, explicit expressions are proved for the covariance matrix of tensors whose entries are invariant under a permutation. These results unlock a whole new class of priors for Bayesian inverse problems. We illustrate how new kernel functions can be designed and efficiently computed and apply our results on two particular Bayesian inverse problems: completing a Hankel matrix from a few noisy measurements and learning an image classifier of handwritten digits. The effectiveness of the proposed priors is demonstrated for both problems. All applications have been implemented as reactive Pluto notebooks in Julia.

Constructing structured tensor priors for Bayesian inverse problems

TL;DR

This paper develops a complete Gaussian prior framework for Bayesian inverse problems that enforces structured tensor solutions through (A,b)-constrained representations. It provides exact mean and covariance characterizations, including explicit constructions for permutation- invariant and Hankel-related structures, and introduces efficient sampling and kernel-based techniques to incorporate these priors in large-scale problems. The approach is demonstrated on Hankel matrix completion and MNIST classifier learning, with practical implementations via reactive Julia Pluto notebooks. The work opens a broad class of structured tensor priors and lays the groundwork for future scalable, kernelized Bayesian inference in high-dimensional tensor settings.

Abstract

Specifying a prior distribution is an essential part of solving Bayesian inverse problems. The prior encodes a belief on the nature of the solution and this regularizes the problem. In this article we completely characterize a Gaussian prior that encodes the belief that the solution is a structured tensor. We first define the notion of (A,b)-constrained tensors and show that they describe a large variety of different structures such as Hankel, circulant, triangular, symmetric, and so on. Then we completely characterize the Gaussian probability distribution of such tensors by specifying its mean vector and covariance matrix. Furthermore, explicit expressions are proved for the covariance matrix of tensors whose entries are invariant under a permutation. These results unlock a whole new class of priors for Bayesian inverse problems. We illustrate how new kernel functions can be designed and efficiently computed and apply our results on two particular Bayesian inverse problems: completing a Hankel matrix from a few noisy measurements and learning an image classifier of handwritten digits. The effectiveness of the proposed priors is demonstrated for both problems. All applications have been implemented as reactive Pluto notebooks in Julia.
Paper Structure (19 sections, 15 theorems, 51 equations, 2 figures, 2 tables, 2 algorithms)

This paper contains 19 sections, 15 theorems, 51 equations, 2 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notation
  3. $(\pmb{A},\pmb{b})$-constrained tensors
  4. Tensors with fixed entries
  5. Known sum of entries
  6. Eigenvector structure
  7. Full characterization of the prior distribution
  8. Recursive nullspace computation
  9. Explicit covariance matrix construction for permutation-invariant tensors
  10. Sparse square root covariance matrix construction for permutation-invariant tensors
  11. Solving the inverse problem
  12. Parameterizing the prior covariance matrix
  13. Change of variables
  14. Structured tensor kernel functions
  15. Applications
  16. ...and 4 more sections

Key Result

Lemma 2.2

\newlabellemma:fixedentries0 Let $\bm{L}$ be the $J(J-1)/2 \times J^2$ matrix that has on each row a single unit entry for each particular occurrence of $j_1-j_2 <(>)\,0$. Lower (upper) triangular tensors are then described by and a vector $\bm{b} \in \mathbb{R}^{\frac{(D-1)(J-1)J^{D-1}}{2}}$ of zeros.

Figures (2)

  • Figure 1: Singular values of the square-root precision matrices of the prior, likelihood and posterior distribution. Only $50\%$ of the Hankel matrix $\bm{W}$ was measured. The noise variance is 1 and the prior variance is $10^{-6}$.
  • Figure 2: Singular values of the square-root precision matrices of the posterior distribution for 4 different priors. The noise variance is fixed to 1.

Theorems & Definitions (46)

  • Definition 2.1
  • Lemma 2.2
  • Proof 1
  • Example 2.3
  • Lemma 2.4
  • Proof 2
  • Example 2.5
  • Definition 2.6
  • Definition 2.7
  • Definition 2.8
  • ...and 36 more