Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nonlinear Bayesian optimal experimental design using logarithmic Sobolev inequalities

Fengyi Li, Ayoub Belhadji, Youssef Marzouk

TL;DR

This work proposes greedy approaches based on new computationally inexpensive lower bounds for MI, constructed via log-Sobolev inequalities, and demonstrates that this method outperforms random selection strategies, Gaussian approximations, and nested Monte Carlo estimators of MI in various settings, including optimal design for nonlinear models with non-additive noise.

Abstract

We study the problem of selecting $k$ experiments from a larger candidate pool, where the goal is to maximize mutual information (MI) between the selected subset and the underlying parameters. Finding the exact solution is to this combinatorial optimization problem is computationally costly, not only due to the complexity of the combinatorial search but also the difficulty of evaluating MI in nonlinear/non-Gaussian settings. We propose greedy approaches based on new computationally inexpensive lower bounds for MI, constructed via log-Sobolev inequalities. We demonstrate that our method outperforms random selection strategies, Gaussian approximations, and nested Monte Carlo (NMC) estimators of MI in various settings, including optimal design for nonlinear models with non-additive noise.

Nonlinear Bayesian optimal experimental design using logarithmic Sobolev inequalities

TL;DR

This work proposes greedy approaches based on new computationally inexpensive lower bounds for MI, constructed via log-Sobolev inequalities, and demonstrates that this method outperforms random selection strategies, Gaussian approximations, and nested Monte Carlo estimators of MI in various settings, including optimal design for nonlinear models with non-additive noise.

Abstract

We study the problem of selecting experiments from a larger candidate pool, where the goal is to maximize mutual information (MI) between the selected subset and the underlying parameters. Finding the exact solution is to this combinatorial optimization problem is computationally costly, not only due to the complexity of the combinatorial search but also the difficulty of evaluating MI in nonlinear/non-Gaussian settings. We propose greedy approaches based on new computationally inexpensive lower bounds for MI, constructed via log-Sobolev inequalities. We demonstrate that our method outperforms random selection strategies, Gaussian approximations, and nested Monte Carlo (NMC) estimators of MI in various settings, including optimal design for nonlinear models with non-additive noise.
Paper Structure (22 sections, 2 theorems, 59 equations, 8 figures, 1 table, 2 algorithms)

This paper contains 22 sections, 2 theorems, 59 equations, 8 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem setup and motivation
  3. The standard greedy approach and challenges
  4. Constructing the bound
  5. The log-Sobolev inequality
  6. Bounding the greedy objective in each iteration
  7. Computing $\nabla_y \log \pi_{X|Y}(x|y)$
  8. Complexity
  9. Related work
  10. Numerical examples
  11. The linear Gaussian problem
  12. Epidemic transmission model
  13. Spatial Poisson process
  14. Conclusion
  15. Acknowledgement
  16. ...and 7 more sections

Key Result

Proposition 1

Let $\mu$ be a probability measure supported on a convex set with $\mathrm{supp}(\mu) \subseteq \mathbb R^p$. We assume $\mu$ admits a Lebesgue density $\rho$ such that $\rho \propto \exp\left(-V- U \right)$, where $V\colon\mathrm{supp}(\mu) \rightarrow \mathbb R$ is a continuous twice differentiabl with the same log-Sobolev constant $\kappa$ for every continuously differentiable function $h$ defi

Figures (8)

  • Figure 1: Epidemic transmission model: (Top) The trajectory of the observations as a function of time. (Bottom) $10$ selected designs using different methods, with a darker color indicating the design being chosen in earlier stages.
  • Figure 2: Epidemic transmission model: MI for designs of increasing size, obtained using LSIG, NMC-greedy, and random selection. Error bars show one standard error, computed using $10$ trials.
  • Figure 3: Spatial Poisson process: The distribution of designs selected for $k = \{4,6,8,10\}$ using different methods, with darker colors indicating more frequent selections over $50$ repetitions.
  • Figure 4: Spatial Poisson process: differences in MI for designs obtained with {LSIG, Gaussian approximation} and with NMC-greedy (i.e., results obtained using NMC-greedy set the zero value at each $k$). MI values for the vertical axis are computed using NMC with $10000$ inner samples and $1000$ outer samples. Error bars show one standard error, computed over $50$ trials.
  • Figure 5: The linear Gaussian problem: The spectrum of $G$, $\Sigma_X$, $\Sigma_\epsilon$ and $\Sigma_Y$.
  • ...and 3 more figures

Theorems & Definitions (2)

  • Proposition 1: Ledoux_logsobGuionnet2003Zahm_CDR
  • Corollary 1