Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diffusion-Shock PDEs for Deep Learning on Position-Orientation Space

Finn M. Sherry, Kristina Schaefer, Remco Duits

Abstract

We extend Regularised Diffusion-Shock (RDS) filtering from Euclidean space $\mathbb{R}_2$ [1] to position-orientation space $\mathbb{M}_2 \cong \mathbb{R}^2 \times S^1$. This has numerous advantages, e.g. making it possible to enhance and inpaint crossing structures, since they become disentangled when lifted to $\mathbb{M}_2$. We create a version of the algorithm using gauge frames to mitigate issues caused by lifting to a finite number of orientations. This leads us to study generalisations of diffusion, since the gauge frame diffusion is not generated by the Laplace-Beltrami operator. RDS filtering compares favourably to existing techniques such as Total Roto-Translational Variation (TR-TV) flow, NLM, and BM3D when denoising images with crossing structures, particularly if they are segmented. Furthermore, we see that $\mathbb{M}_2$ RDS inpainting is indeed able to restore crossing structures, unlike $\mathbb{R}^2$ RDS inpainting. In addition to the contributions of our SSVM submission "Diffusion-Shock Filtering on the Space of Positions and Orientations", in this extended work we provide new theorical results and automate RDS filtering by integrating it into a geometric deep learning framework. Regarding our theoretical contributions, we prove that our generalised diffusions are still well-posed, smoothing, and analytic. We developed an RDS filtering PDE layer for the PDE-CNN and PDE-G-CNN deep learning frameworks, using a novel gating mechanism. We show that these new RDS PDE layers can be beneficial in various impainting and denoising tasks.

Diffusion-Shock PDEs for Deep Learning on Position-Orientation Space

Abstract

We extend Regularised Diffusion-Shock (RDS) filtering from Euclidean space [1] to position-orientation space . This has numerous advantages, e.g. making it possible to enhance and inpaint crossing structures, since they become disentangled when lifted to . We create a version of the algorithm using gauge frames to mitigate issues caused by lifting to a finite number of orientations. This leads us to study generalisations of diffusion, since the gauge frame diffusion is not generated by the Laplace-Beltrami operator. RDS filtering compares favourably to existing techniques such as Total Roto-Translational Variation (TR-TV) flow, NLM, and BM3D when denoising images with crossing structures, particularly if they are segmented. Furthermore, we see that RDS inpainting is indeed able to restore crossing structures, unlike RDS inpainting. In addition to the contributions of our SSVM submission "Diffusion-Shock Filtering on the Space of Positions and Orientations", in this extended work we provide new theorical results and automate RDS filtering by integrating it into a geometric deep learning framework. Regarding our theoretical contributions, we prove that our generalised diffusions are still well-posed, smoothing, and analytic. We developed an RDS filtering PDE layer for the PDE-CNN and PDE-G-CNN deep learning frameworks, using a novel gating mechanism. We show that these new RDS PDE layers can be beneficial in various impainting and denoising tasks.

Paper Structure

This paper contains 29 sections, 10 theorems, 113 equations, 14 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Smoothing and Shock Filtering
  4. Multi-Orientation Image Processing
  5. PDE-Based Convolutional Neural Networks
  6. Our Contributions
  7. Structure of the Article
  8. Preliminaries
  9. Multi-Orientation Processing
  10. Gauge Frames
  11. Regularised Diffusion-Shock Filtering on Euclidean Space
  12. Regularised Diffusion-Shock Filtering on the Space of Positions & Orientations
  13. Generalised Laplacians
  14. PDE Scheme
  15. Experimental Results
  16. ...and 14 more sections

Key Result

Theorem 8

Let $G$ be a connected Lie group, let $\mathcal{G}$ be an invariant metric thereon, and let $\nu \in \mathbb{R}$. With respect to an invariant frame $\{\mathcal{A}_i\}_i$, the Lie-Cartan Laplacian is given by while the Laplace-Beltrami operator is given by with $\Gamma_{ij}^k$ the Christoffel symbols given by $\Gamma_{ij}^k \mathcal{A}_k = \nabla_{\mathcal{A}_i}^\mathrm{LC} \mathcal{A}_j$ and $c

Figures (14)

  • Figure 1: Performing multi-orientation processing. Lifting disentangles crossing and overlapping structures.
  • Figure 2: Schematic PDE-G-CNN. In PDE-CNNs, the 'Lift' and 'Project' layers can be omitted. In each layer, every feature map evolves according to a specified PDE with channel-dependent trainable parameters. Subsequently, the feature maps are mixed affinely, where the mixing weights are also trainable.
  • Figure 3: Isosurfaces of the exact Riemannian distance and its logarithmic aproximation on $\mathbb{M}_2$ with spatial anisotropy $\zeta = \frac{1}{3}$. In PDE-G-CNNs, the shape of these isosurfaces are learned.
  • Figure 4: (a) Cake wavelet for orientation $\theta = \pi / 8$. (b-d) Isosurfaces of invariant Riemannian distances with varying spatial anisotropies $\zeta \coloneqq \sqrt{g_{11} / g_{22}}$.
  • Figure 5: Comparison of gauge frame and standard left-invariant frame. From the top view (a) we see that $\mathcal{A}_1^U$ has been rotated towards $\mathcal{A}_2$ to compensate for the deviation from horizontality $d_H$. From the side view (b) we see that $\mathcal{A}_1^U$ has been rotated towards $\mathcal{A}_3$; the rotation angle is related to the curvature $\kappa$.
  • ...and 9 more figures

Theorems & Definitions (34)

  • Definition 1: Position-orientation space
  • Definition 2: Special Euclidean Group SE(2)
  • Definition 3: Invariant Frame
  • Definition 4: Orientation Score
  • Remark 1
  • Definition 5: First Gauge Vector
  • Remark 2
  • Definition 6: Generalised Laplacian
  • Definition 7: Lie-Cartan Connection
  • Remark 3
  • ...and 24 more