Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gradient Flow for Finding E-optimal Designs

Jieling Shi, Kim-Chuan Toh, Xin T. Tong, Weng Kee Wong

TL;DR

The paper tackles the challenging E-optimal design problem, which seeks to maximize the smallest eigenvalue of the information matrix and is non-differentiable when the smallest eigenvalue has multiplicity greater than one. It develops a Wasserstein gradient-flow approach on the space of probability measures to solve design optimization, deriving explicit gradients in the simple-eigenvalue case and introducing a Wasserstein steepest ascent direction for non-simple eigenvalues via an SDP relaxation. By connecting infinite-dimensional gradient dynamics to finite-particle systems, the authors provide theoretical guarantees for the particle approximation and demonstrate competitive numerical performance on both linear and nonlinear models, including constrained designs via projected Wasserstein flows. The results show near-optimal E-designs with favorable convergence properties and computational scalability compared to heuristic methods, highlighting optimal transport-based dynamics as a unifying framework for challenging design problems. This framework extends naturally to other design criteria and constrained settings, offering a principled, geometry-driven path for optimal experimental design.

Abstract

We investigate the use of Wasserstein gradient flows for finding an $E$-optimal design for a regression model. Unlike the commonly used $D$- and $L$-optimality criteria, the $E$-criterion finds a design that maximizes the smallest eigenvalue of the information matrix, and so it is a non-differentiable criterion unless the minimum eigenvalue has geometric multiplicity equals to one. Such maximin design problems abound in statistical applications and present unique theoretical and computational challenges. Building on the differential structure of the $2$-Wasserstein space, we derive explicit formulas for the Wasserstein gradient of the $E$-optimality criterion in the simple-eigenvalue case. For higher multiplicities, we propose a Wasserstein steepest ascent direction and show that it can be computed exactly via a semidefinite programming (SDP) relaxation. We develop particle approximations that connect infinite-dimensional flows with finite-dimensional optimization, and provide approximation guarantees for empirical measures. Our framework extends naturally to constrained designs via projected Wasserstein gradient flows. Numerical experiments demonstrate that the proposed methods successfully recover $E$-optimal designs for both linear and nonlinear regression models, with competitive accuracy and scalability compared to existing heuristic approaches. This work highlights the potential of optimal transport-based dynamics as a unifying tool for studying challenging optimal design problems.

Gradient Flow for Finding E-optimal Designs

TL;DR

The paper tackles the challenging E-optimal design problem, which seeks to maximize the smallest eigenvalue of the information matrix and is non-differentiable when the smallest eigenvalue has multiplicity greater than one. It develops a Wasserstein gradient-flow approach on the space of probability measures to solve design optimization, deriving explicit gradients in the simple-eigenvalue case and introducing a Wasserstein steepest ascent direction for non-simple eigenvalues via an SDP relaxation. By connecting infinite-dimensional gradient dynamics to finite-particle systems, the authors provide theoretical guarantees for the particle approximation and demonstrate competitive numerical performance on both linear and nonlinear models, including constrained designs via projected Wasserstein flows. The results show near-optimal E-designs with favorable convergence properties and computational scalability compared to heuristic methods, highlighting optimal transport-based dynamics as a unifying framework for challenging design problems. This framework extends naturally to other design criteria and constrained settings, offering a principled, geometry-driven path for optimal experimental design.

Abstract

We investigate the use of Wasserstein gradient flows for finding an -optimal design for a regression model. Unlike the commonly used - and -optimality criteria, the -criterion finds a design that maximizes the smallest eigenvalue of the information matrix, and so it is a non-differentiable criterion unless the minimum eigenvalue has geometric multiplicity equals to one. Such maximin design problems abound in statistical applications and present unique theoretical and computational challenges. Building on the differential structure of the -Wasserstein space, we derive explicit formulas for the Wasserstein gradient of the -optimality criterion in the simple-eigenvalue case. For higher multiplicities, we propose a Wasserstein steepest ascent direction and show that it can be computed exactly via a semidefinite programming (SDP) relaxation. We develop particle approximations that connect infinite-dimensional flows with finite-dimensional optimization, and provide approximation guarantees for empirical measures. Our framework extends naturally to constrained designs via projected Wasserstein gradient flows. Numerical experiments demonstrate that the proposed methods successfully recover -optimal designs for both linear and nonlinear regression models, with competitive accuracy and scalability compared to existing heuristic approaches. This work highlights the potential of optimal transport-based dynamics as a unifying tool for studying challenging optimal design problems.
Paper Structure (18 sections, 10 theorems, 135 equations, 3 figures, 2 tables, 2 algorithms)

This paper contains 18 sections, 10 theorems, 135 equations, 3 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. General Framework, First-Order Conditions, and Particle Approximation
  3. Wasserstein Geometry, Gradients, and Gradient Flows
  4. Particle Wasserstein Gradient Flow
  5. Particle Wasserstein Gradient Flow for the E-Optimal Design Problem
  6. Derivation of the Wasserstein steepest ascent
  7. Necessity condition
  8. SDP relaxation
  9. Numerical experiments
  10. D-optimal design
  11. E-optimal design
  12. Second-order response surface model
  13. Logistic regression model
  14. Conclusions
  15. Proof of the Lemma and Propositions
  16. ...and 3 more sections

Key Result

Proposition 2.1

Let $\mathcal{F}:\mathcal{P}_2(\mathbb{R}^d)\to\mathbb{R}$ be Wasserstein differentiable at $\rho^\star$, with Wasserstein gradient $\nabla_{W_2}\mathcal{F}(\rho^\star)$$\in \operatorname{Tan}_{\rho^\star}\mathcal{P}_2(\mathbb{R}^d)$ in the sense of Definition def:Wassersteingrad. If $\rho^\star$ is and in particular

Figures (3)

  • Figure 1: Convergence curves for the $D$-optimal design problem obtained by the particle Wasserstein gradient flow (WGF) and particle swarm optimization (PSO) methods under different constraints. The PSO curve is averaged over 100 independent runs.
  • Figure 2: Convergence curves for the $E$-optimal design problem for the second-order response surface model under different dimensions and constraints, obtained by the particle Wasserstein gradient flow (WGF) and particle swarm optimization (PSO) methods. The PSO curve is averaged over 100 independent runs.
  • Figure 3: Convergence curves for the $E$-optimal design problem in the logistic model, computed via particle Wasserstein gradient flow (WGF) and particle swarm optimization (PSO). The PSO curve is averaged over 100 independent runs, while the WGF curve is smoothed by averaging the objective values over every 10 iterations.

Theorems & Definitions (25)

  • Definition 2.1: Wasserstein distance
  • Definition 2.2: tangent space ambrosio_gradient_2008
  • Definition 2.3: Wasserstein differentiable functionals Lanzetti2025FirstOrder
  • Proposition 2.1: First-order optimality condition in $\mathcal{P}_2(\mathbb{R}^d)$ Lanzetti2025FirstOrder
  • Definition 2.4: Wasserstein gradient flow ambrosio_gradient_2008
  • Proposition 2.2
  • Corollary 2.1
  • Proposition 2.3
  • proof
  • Remark 2.1
  • ...and 15 more