Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PASOA- PArticle baSed Bayesian Optimal Adaptive design

Jacopo Iollo, Christophe Heinkelé, Pierre Alliez, Florence Forbes

TL;DR

A novel combination of stochastic optimization and tempered SMC allows to jointly handle design optimization and parameter inference, and a proof that the obtained optimal design estimators benefit from some consistency property is provided.

Abstract

We propose a new procedure named PASOA, for Bayesian experimental design, that performs sequential design optimization by simultaneously providing accurate estimates of successive posterior distributions for parameter inference. The sequential design process is carried out via a contrastive estimation principle, using stochastic optimization and Sequential Monte Carlo (SMC) samplers to maximise the Expected Information Gain (EIG). As larger information gains are obtained for larger distances between successive posterior distributions, this EIG objective may worsen classical SMC performance. To handle this issue, tempering is proposed to have both a large information gain and an accurate SMC sampling, that we show is crucial for performance. This novel combination of stochastic optimization and tempered SMC allows to jointly handle design optimization and parameter inference. We provide a proof that the obtained optimal design estimators benefit from some consistency property. Numerical experiments confirm the potential of the approach, which outperforms other recent existing procedures.

PASOA- PArticle baSed Bayesian Optimal Adaptive design

TL;DR

A novel combination of stochastic optimization and tempered SMC allows to jointly handle design optimization and parameter inference, and a proof that the obtained optimal design estimators benefit from some consistency property is provided.

Abstract

We propose a new procedure named PASOA, for Bayesian experimental design, that performs sequential design optimization by simultaneously providing accurate estimates of successive posterior distributions for parameter inference. The sequential design process is carried out via a contrastive estimation principle, using stochastic optimization and Sequential Monte Carlo (SMC) samplers to maximise the Expected Information Gain (EIG). As larger information gains are obtained for larger distances between successive posterior distributions, this EIG objective may worsen classical SMC performance. To handle this issue, tempering is proposed to have both a large information gain and an accurate SMC sampling, that we show is crucial for performance. This novel combination of stochastic optimization and tempered SMC allows to jointly handle design optimization and parameter inference. We provide a proof that the obtained optimal design estimators benefit from some consistency property. Numerical experiments confirm the potential of the approach, which outperforms other recent existing procedures.
Paper Structure (38 sections, 10 theorems, 69 equations, 9 figures, 3 tables, 3 algorithms)

This paper contains 38 sections, 10 theorems, 69 equations, 9 figures, 3 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Sequential Bayesian Optimal Experimental Design (BOED)
  4. Expected Information Gain (EIG).
  5. Sequential design.
  6. EIG contrastive bound optimization.
  7. Tempered Sequential Monte Carlo
  8. Particle EIG contrastive bound approximation and optimization
  9. Asymptotic properties
  10. Convergence of product particle approximations.
  11. Consistency of the sequential design estimators.
  12. Numerical experiments
  13. Source location.
  14. Constant Elasticity of Substitution (CES).
  15. Conclusion
  16. ...and 23 more sections

Key Result

Proposition 1

Using Algorithm alg:SMC with multinomial resampling and assuming that all potential functions ${G}_{k,\tau}$ are upper bounded, there exists a constant $c_k$ such that, for all functions $\phi \in {\cal C}_b({\boldsymbol{\Theta}}\xspace^{L+1})$, with $c_{N,L} = 1- (1 -\frac{1}{N})^{L+1} \approx \frac{L+1}{N}$ and where the expectation is taken over all the realizations of the random tempered SMC

Figures (9)

  • Figure 1: Source location example at steps 5, 7, 9, 11. Over design steps, particles concentrate faster to true sources (red crosses) with PASOA (2nd line) than with SMC (1st line). Lower particle weights in blue, higher in yellow.
  • Figure 2: Tempered SMC, a SMC step from $p_{k-1}$ to $p_k$ (blue) performed in ${\cal T}$ intermediate tempering steps (red).
  • Figure 3: Source location. Column 1: median and standard error over 100 rollouts for SPCE (top), SNMC (middle) and L$_2$ Wasserstein distance (bottom) with respect to the number of experiments $k$. Column 2: impact of the number of particles (5K to 1M) on median SPCE, SNMC and Wasserstein distance for PASOA (plain) and SMC (dotted). Note the logarithmic scale.
  • Figure 4: CES example. Median and standard error over 100 rollouts, with respect to the number of experiments $k$, for SPCE -plain , SNMC -dashed (left) and Wasserstein distance (right).
  • Figure 5: PASOA evolution of particles (in purple) over some selected steps $k$. Particles correspond initially to a sample from the prior $p({\boldsymbol{\theta}}\xspace)$ and progressively evolve to a sample of particles located around the initially unknown true source positions indicated by red crosses. Green crosses indicate the optimal measurement locations ${\boldsymbol{\xi}}\xspace_k^*$ obtained at each step $k$.
  • ...and 4 more figures

Theorems & Definitions (10)

  • Proposition 1: L$_2$ convergence
  • Proposition 2: Consistency
  • Lemma 1
  • Lemma 2: Monte Carlo error
  • Lemma 3: Weights normalization error
  • Proposition 1: L$_2$ convergence
  • Theorem 1: Theorem 5.7 in vaart_1998
  • Lemma 4
  • Lemma 5
  • Proposition 2