Spectral Function Space Learning and Numerical Linear Algebra Networks for Solving Linear Inverse Problems

TL;DR

The paper presents a data-driven framework for solving linear inverse problems by learning a possibly ill-conditioned operator $F$ from training pairs via a two-step encoding: Gram–Schmidt orthonormalization of input samples and PCA on the resulting outputs, yielding a spectral decomposition $F= V D U^T$. Both steps are shown to admit customized neural-network implementations, forming a linear algebra network that enables stable, data-driven inversion without relying on a physical forward model. The decoder computes a minimum-norm preimage by projecting onto the PCA-derived spectral basis and mapping back through $F^*$, with theory linking recovered vectors to the operator’s singular subspaces. Numerical experiments with the Radon transform demonstrate that the method can recover spectral functions and provide effective decoding, validating the approach as a data-driven tool for spectral analysis and inverse problem solving in settings where the forward operator is learned from data.

Abstract

We consider solving a probably ill-conditioned linear operator equation, where the operator is not modeled by physical laws but is specified via training pairs (consisting of images and data) of the input-output relation of the operator. We derive a stable method for computing the operator, which consists of first a Gram-Schmidt orthonormalization of images and a principal component analysis of the data. This two-step algorithm provides a spectral decomposition of the linear operator. Moreover, we show that both Gram-Schmidt and principal component analysis can be written as a deep neural network, which relates this procedure to de-and encoder networks. Therefore, we call the two-step algorithm a linear algebra network. Finally, we provide numerical simulations showing the strategy is feasible for reconstructing spectral functions and for solving operator equations without explicitly exploiting the physical model.

Figures (10)

• Figure 1: Variational en- and decoding with neural networks: The image data are represented via a neural network. After that they are transformed into a feature space (with the operator $\mathcal{N}_{X}$). The features are compressed by a principal geodesic analysis. The decoder $\mathcal{N}_{Y}^{-1}$ (we assume for the sake of simplicity that the operator is invertible) transforms features into images.
• Figure 2: En- and decoding scheme for linear operators: First the image data are orthonormalized and the according data is computed by applying $F$ - this is done by explicit calculations without making use on any physical model describing the forward operator. On the orthonormalized data a principal component analysis (PCA) is applied, which allows to compress the data space. The decoder calculates the inverse of some given data in the compressed space.
• Figure 3: The neural network structure of the Gram-Schmidt orthonormalization. We use here the function $\sigma_\varepsilon$ from \ref{['eq:sigmave']} as activation function.
• Figure 4: This plot visualizes the first twenty normalized singular functions $v_{kl}$, defined in \ref{['eq:vkl_ukl']}, for $k =0,1,\ldots,7$, $l=0,1,\ldots,k$ and $l+k$ even.
• Figure 5: This plot visualizes the first twenty singular functions $u_{k,l}$, defined in \ref{['eq:vkl_ukl']}, for $k =0,1,\ldots,7$, $l=0,1,\ldots,k$ and $l+k$ even.

Theorems & Definitions (19)

• Remark 1
• Remark 2
• Theorem 1: Spectral theory: See Theorem 2.5.2 in GolVan96
• Definition 1
• Definition 2
• Theorem 2
• Proof 1
• Remark 3
• Remark 4
• Lemma 3.1: Decoder