Unique reconstruction for discretized inverse problems: a random sketching approach via subsampling
Ruhui Jin, Qin Li, Anjali Nair, Samuel Stechmann
TL;DR
This work studies parameter reconstruction in discretized inverse problems under limited data, introducing a random sketching approach that reduces the Gauss-Newton Hessian to a small $m\times m$ matrix ${\bf H}^N_s$ via uniform row sampling. It proves that, for rank $r$ and with $m\le r$, the sketched Hessian is well-conditioned with high probability, specifically $\mathbb{P}\left(\mathrm{cond}(\mathbf{H}^N_s) \le \frac{L+\tau(m)}{\ell-\tau(m)}\right) \ge 1-\frac{1}{r}$, where $\ell$ and $L$ bound the diagonal of ${\bf H}^N$ and $\tau(m)$ is a distortion term dependent on $m$, $r$, the coherence $\mu$, and norms of ${\bf H}^N$. The proof leverages matrix concentration inequalities and a moment bound on the hollow Gram component ${\bf M}^N_s={\bf H}^N_s-\mathrm{diag}(\mathbf{H}^N_s)$, yielding a sub-Gaussian tail. Numerical experiments on synthetic data and a PDE-based elliptic inverse problem corroborate the theory, showing that when $m$ scales with the data rank $r$, the reduced Hessian remains well-conditioned with high probability, enabling stable, locally unique reconstructions in the discrete setting.
Abstract
Theoretical inverse problems are often studied in an ideal infinite-dimensional setting. The well-posedness theory provides a unique reconstruction of the parameter function, when an infinite amount of data is given. Through the lens of PDE-constrained optimization, this means one attains the zero-loss property of the mismatch function in this setting. This is no longer true in computations when we are limited to finite amount of measurements due to experimental or economical reasons. Consequently, one must compromise the goal, from inferring a function, to a discrete approximation. What is the reconstruction power of a fixed number of data observations? How many parameters can one reconstruct? Here we describe a probabilistic approach, and spell out the interplay of the observation size $(r)$ and the number of parameters to be uniquely identified $(m)$. The technical pillar here is the random sketching strategy, in which the matrix concentration inequality and sampling theory are largely employed. By analyzing a randomly subsampled Hessian matrix, we attain a well-conditioned reconstruction problem with high probability. Our main theory is validated in numerical experiments, using an elliptic inverse problem as an example.