Neural Optimal Design of Experiment for Inverse Problems

TL;DR

NODE addresses optimal experimental design for inverse problems by jointly learning a neural reconstruction model with a fixed-budget, continuous design of sensor locations, thereby eliminating nested bi-level optimization and indirect sparsity regularization. By optimizing actual sensor placements rather than grid weights and employing interpolation to bridge continuous and discrete design spaces, NODE achieves compact, informative designs without $oldsymbol{l}^1$ tuning and with reduced computational cost. The framework is validated on an analytically tractable exponential-growth benchmark, MNIST sampling, and sparse-view X-ray CT, where NODE consistently improves reconstruction accuracy and task performance compared to baselines. Adaptivity is integrated to support sequential experiments, and results show NODE captures meaningful geometric structure in the inverse problems, suggesting significant practical impact for efficient data acquisition in imaging and beyond.

Abstract

We introduce Neural Optimal Design of Experiments, a learning-based framework for optimal experimental design in inverse problems that avoids classical bilevel optimization and indirect sparsity regularization. NODE jointly trains a neural reconstruction model and a fixed-budget set of continuous design variables representing sensor locations, sampling times, or measurement angles, within a single optimization loop. By optimizing measurement locations directly rather than weighting a dense grid of candidates, the proposed approach enforces sparsity by design, eliminates the need for l1 tuning, and substantially reduces computational complexity. We validate NODE on an analytically tractable exponential growth benchmark, on MNIST image sampling, and illustrate its effectiveness on a real world sparse view X ray CT example. In all cases, NODE outperforms baseline approaches, demonstrating improved reconstruction accuracy and task-specific performance.

Figures (9)

• Figure 1: Evolution of sensor location and training loss during NODE training. NODE is applied to \ref{['eq:exp_inverse_solution']} for $m=3$ sensors. Prior distributions for model parameters are $a\sim\mathcal{U}(0.5,1.5)$ and $b\sim\mathcal{U}(1,2)$. Locations and model weights are trained simultaneously, using Adam optimizer, with learning rates of $10^{-1}$ and $10^{-3}$, respectively. Training lasts for 4000.0 epochs with batch size 1024.0. Sensor locations stabilize to either 0 or 1 at the same time that the loss stabilizes.
• Figure 2: Optimal placement of design points at the boundary locations $t=0$ and $t=1$. Shown is the ratio of points assigned to each endpoint for increasing total design size $m$. The optimal design always places all sampling locations on the boundary, and the discrete optimal ratios converge as $m\to\infty$ to the continuous limits $\sqrt{2}-1 \approx 41.42\%$ at $t=1$ and $2-\sqrt{2} \approx 58.58\%$ at $t=0$.
• Figure 3: Neural OED convergence to the theoretical endpoint design for the scalar exponential model $y(t)=b\,\exp(a t)$. For each $m=2,\dots, 200$ we train the NODE algorithm (\ref{['alg:node']}) with batch size $1024$, $10{,}000$ Adam steps (learning rates $10^{-3}$ for the reconstruction network and $10^{-1}$ for the design variables), Gaussian noise level $\varepsilon=0.05$ on the log-transformed data, and a single-hidden-layer ReLU MLP of width $256$ that reconstructs $(a,b)$ from noisy measurements and their locations. The plot shows, for each $m$, the fractions $k_1/m$ (blue, points near $t=1$) and $k_0/m$ (orange, points near $t=0$), together with the theoretical limits $\sqrt{2}-1\approx 0.4142$ and $2-\sqrt{2}\approx 0.5858$ (dashed lines). The learned designs systematically concentrate on the endpoints and the empirical fractions cluster around the theoretical values, demonstrating that the neural OED recovers both the qualitative endpoint structure and the optimal asymptotic splitting of design points.
• Figure 4: Sample image reconstructions of images, representing the digits 4,0,6. The top row displays the true images. Below, reconstructions are shown for different models. Models use NODE designs, random designs, or high variance designs, are trained using either the MSE loss \ref{['equ:mnist_mse_nn']} or max loss \ref{['equ:mnist_max_nn']}, and their designs use $M=10,50,100$ sensors.
• Figure 5: Log-scale comparison of the distribution of reconstruction error over the testing set of 10000.0 images. Top figure shows reconstruction errors under the MSE loss \ref{['equ:mnist_mse_nn']}. Bottom figure shows reconstructions errors under the max loss \ref{['equ:mnist_max_nn']}. Distributions are computed different neural network architectures making use of $M=1,10,20,30,40,50,60,70,80,90,100$ sensor locations. Green distributions correspond to designs where locations are optimized by NODE. Blue distributions correspond to designs where locations fixed at randomly chosen values. Red distributions correspond to designs that use locations with the highest variance across the training set.