Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Geometric Approach to Optimal Experimental Design

Gavin Kerrigan, Christian A. Naesseth, Tom Rainforth

TL;DR

This work introduces mutual transport dependence (MTD), a geometric, cost-based optimal transport objective for Bayesian experimental design that replaces density-based information measures. By defining $ ext{T}_c(d)={ m OT}_c[p( heta,y|d),p( heta)p(y|d)]$ and leveraging sample-based estimation, it enables design optimization without nested density evaluations and can incorporate downstream error metrics through the choice of cost $c$. The authors establish theoretical connections to mutual information, provide practical estimation and gradient-based optimization procedures, and demonstrate via CES and source-location experiments that MTD can outperform traditional MI-based design under various cost specifications and transformations. Overall, the framework offers a flexible, robust alternative to information-theoretic OED, adaptable to downstream objectives and implicit likelihoods with competitive computational efficiency.

Abstract

We introduce a novel geometric framework for optimal experimental design (OED). Traditional OED approaches, such as those based on mutual information, rely explicitly on probability densities, leading to restrictive invariance properties. To address these limitations, we propose the mutual transport dependence (MTD), a measure of statistical dependence grounded in optimal transport theory which provides a geometric objective for optimizing designs. Unlike conventional approaches, the MTD can be tailored to specific downstream estimation problems by choosing appropriate geometries on the underlying spaces. We demonstrate that our framework produces high-quality designs while offering a flexible alternative to standard information-theoretic techniques.

A Geometric Approach to Optimal Experimental Design

TL;DR

This work introduces mutual transport dependence (MTD), a geometric, cost-based optimal transport objective for Bayesian experimental design that replaces density-based information measures. By defining and leveraging sample-based estimation, it enables design optimization without nested density evaluations and can incorporate downstream error metrics through the choice of cost . The authors establish theoretical connections to mutual information, provide practical estimation and gradient-based optimization procedures, and demonstrate via CES and source-location experiments that MTD can outperform traditional MI-based design under various cost specifications and transformations. Overall, the framework offers a flexible, robust alternative to information-theoretic OED, adaptable to downstream objectives and implicit likelihoods with competitive computational efficiency.

Abstract

We introduce a novel geometric framework for optimal experimental design (OED). Traditional OED approaches, such as those based on mutual information, rely explicitly on probability densities, leading to restrictive invariance properties. To address these limitations, we propose the mutual transport dependence (MTD), a measure of statistical dependence grounded in optimal transport theory which provides a geometric objective for optimizing designs. Unlike conventional approaches, the MTD can be tailored to specific downstream estimation problems by choosing appropriate geometries on the underlying spaces. We demonstrate that our framework produces high-quality designs while offering a flexible alternative to standard information-theoretic techniques.
Paper Structure (40 sections, 11 theorems, 81 equations, 10 figures, 1 table)

This paper contains 40 sections, 11 theorems, 81 equations, 10 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background and Notation
  3. OED with Mutual Information
  4. Optimal Transport
  5. Experimental Design through Mutual Transport Dependence
  6. Mutual Transport Dependence
  7. Alternative Transport-Based Measures
  8. Estimation and Optimization
  9. Estimation.
  10. Optimization.
  11. Comparing Mutual Information and Mutual Transport Dependence
  12. Related Work
  13. Experiments
  14. CES.
  15. Source Location Finding.
  16. ...and 25 more sections

Key Result

Theorem 1

For a fixed $1 \leq p < \infty$, consider the cost for $\eta > 0$. Then, $\eta^{-1} \mathcal{T}_{c_\eta}(d) \to \mathcal{T}_{c_\Theta}^{{(\theta)}}(d)$ as $\eta \to 0^{+}$, where $c_\Theta(\theta, \theta') = |\theta - \theta'|^p_\Theta$. Similarly, for we have $\psi^{-1}\mathcal{T}_{c_\psi}(d) \to \mathcal{T}_{c_\mathcal{Y}}^{(y)}(d)$ as $\psi \to 0^{+}$ where $c_\mathcal{Y}(y, y') = |y - y'|^p_

Figures (10)

  • Figure 1: Comparison of and on the 1D source location-finding problem, with the true source at $\theta=-1$ (dashed red line). MI is maximized at the origin, reflecting the prior mode, whereas is maximized near $d \approx \pm 1.3$. The posterior for $d=0$ is bimodal, while for $d=\pm 1.3$ it becomes unimodal or sharply concentrated. In practice, $d$ is optimized directly, and breaks the posterior symmetry even when initialized unfavorably.
  • Figure 2: RMSE between posterior $\theta$ samples and ground-truth on the location finding problem, averaged over 25 seeds ($\pm$ one standard error).
  • Figure 3: Estimated values of the EIG (via PCE) and the for the 2D location-finding model with two sources. Top row: the EIG is maximized at the origin, whereas the MTD favors off-center designs, illustrating its symmetry-breaking behavior. Bottom row: with additional weighting of the cost function, the can be tuned to favor specific regions of the design space.
  • Figure 4: Expected zero-one loss for the 2D location-finding model. The weighted rapidly determines whether $\theta \in \mathcal{R}$.
  • Figure 5: Values of the and for the linear-Gaussian model for $\sigma^2_{d,\theta} = 1$.
  • ...and 5 more figures

Theorems & Definitions (22)

  • Definition 1
  • Definition 2
  • Definition 3
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Definition 4: Triangular Couplings
  • Lemma 1
  • proof
  • Theorem 4
  • ...and 12 more