Equivariant Neural Networks for Indirect Measurements

TL;DR

This paper proposes a class of equivariant neural networks that can be directly applied to the measurements to solve the desired task and rigorously analyze the relation between the measurement operator and the resulting group representations and proves a representer theorem that characterizes the class of linear operators that translate between a given pair of group actions.

Abstract

In recent years, deep learning techniques have shown great success in various tasks related to inverse problems, where a target quantity of interest can only be observed through indirect measurements by a forward operator. Common approaches apply deep neural networks in a post-processing step to the reconstructions obtained by classical reconstruction methods. However, the latter methods can be computationally expensive and introduce artifacts that are not present in the measured data and, in turn, can deteriorate the performance on the given task. To overcome these limitations, we propose a class of equivariant neural networks that can be directly applied to the measurements to solve the desired task. To this end, we build appropriate network structures by developing layers that are equivariant with respect to data transformations induced by well-known symmetries in the domain of the forward operator. We rigorously analyze the relation between the measurement operator and the resulting group representations and prove a representer theorem that characterizes the class of linear operators that translate between a given pair of group actions. Based on this theory, we extend the existing concepts of Lie group equivariant deep learning to inverse problems and introduce new representations that result from the involved measurement operations. This allows us to efficiently solve classification, regression or even reconstruction tasks based on indirect measurements also for very sparse data problems, where a classical reconstruction-based approach may be hard or even impossible. We illustrate the effectiveness of our approach in numerical experiments and compare with existing methods.

Figures (7)

• Figure 1: The setting of our approach, illustrated for 2-angle fan-beam Radon measurements of tubes: symmetries in $\mathcal{Y}$ (right) are induced by well-known group representations in the physical space $\mathcal{X}$ (left). However, the data is measured only at a finite set of sensor points $V$, depicted by gray circles.
• Figure 1: Illustration how classical reconstructions (right) of an MNIST digit based on sparse measurements (center, visualized by circles) behave under transformations from $\mathrm{Aff}^{+}(2)=\mathrm{T}(2)\rtimes\mathrm{GL}^+(2)$ (left). In this case, the resulting representation in the full sinograms (center) is a generalized domain transform.
• Figure 1: Overview of our approach, including the measurement scheme as well as the learned processing by an equivariant neural network. The plots illustrate the modelling and symmetries of a two-angle fan-beam Radon operator on a rotated and translated elliptical ring.
• Figure 1: Classification accuracy for fan-beam CT measurements of randomly rotated or $\mathrm{GL}^+(2)$-transformed MNIST test dataset for different neural networks that are trained exclusively on upright digits.
• Figure 2: Classification accuracy for fan-beam CT projections of the RotMNIST dataset for different numbers of measurement angles when trained on RotMNIST and tested on RotMNIST or LinMNIST.

Theorems & Definitions (6)

• Theorem 3.1: visibility condition
• Proof 1
• Theorem 3.2: Equivariant measurement operator is a convolution.
• Remark 3.3
• Proof 2: Proof of Theorem \ref{['thm:equivariance']}
• Example 3.4