Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unsupervised Learning of the Total Variation Flow

Tamara G. Grossmann, Sören Dittmer, Yury Korolev, Carola-Bibiane Schönlieb

TL;DR

The TVflowNET is proposed, an unsupervised neural network approach, to approximate the solution of the TV flow given an initial image and a time instance, which requires no ground truth data but rather makes use of the PDE for optimisation of the network parameters.

Abstract

The total variation (TV) flow generates a scale-space representation of an image based on the TV functional. This gradient flow observes desirable features for images, such as sharp edges and enables spectral, scale, and texture analysis. Solving the TV flow is challenging; one reason is the the non-uniqueness of the subgradients. The standard numerical approach for TV flow requires solving multiple non-smooth optimisation problems. Even with state-of-the-art convex optimisation techniques, this is often prohibitively expensive and strongly motivates the use of alternative, faster approaches. Inspired by and extending the framework of physics-informed neural networks (PINNs), we propose the TVflowNET, an unsupervised neural network approach, to approximate the solution of the TV flow given an initial image and a time instance. The TVflowNET requires no ground truth data but rather makes use of the PDE for optimisation of the network parameters. We circumvent the challenges related to the non-uniqueness of the subgradients by additionally learning the related diffusivity term. Our approach significantly speeds up the computation time and we show that the TVflowNET approximates the TV flow solution with high fidelity for different image sizes and image types. Additionally, we give a full comparison of different network architecture designs as well as training regimes to underscore the effectiveness of our approach.

Unsupervised Learning of the Total Variation Flow

TL;DR

The TVflowNET is proposed, an unsupervised neural network approach, to approximate the solution of the TV flow given an initial image and a time instance, which requires no ground truth data but rather makes use of the PDE for optimisation of the network parameters.

Abstract

The total variation (TV) flow generates a scale-space representation of an image based on the TV functional. This gradient flow observes desirable features for images, such as sharp edges and enables spectral, scale, and texture analysis. Solving the TV flow is challenging; one reason is the the non-uniqueness of the subgradients. The standard numerical approach for TV flow requires solving multiple non-smooth optimisation problems. Even with state-of-the-art convex optimisation techniques, this is often prohibitively expensive and strongly motivates the use of alternative, faster approaches. Inspired by and extending the framework of physics-informed neural networks (PINNs), we propose the TVflowNET, an unsupervised neural network approach, to approximate the solution of the TV flow given an initial image and a time instance. The TVflowNET requires no ground truth data but rather makes use of the PDE for optimisation of the network parameters. We circumvent the challenges related to the non-uniqueness of the subgradients by additionally learning the related diffusivity term. Our approach significantly speeds up the computation time and we show that the TVflowNET approximates the TV flow solution with high fidelity for different image sizes and image types. Additionally, we give a full comparison of different network architecture designs as well as training regimes to underscore the effectiveness of our approach.
Paper Structure (18 sections, 4 theorems, 19 equations, 8 figures, 9 tables, 1 algorithm)

This paper contains 18 sections, 4 theorems, 19 equations, 8 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions
  3. Mathematical Background
  4. Theory
  5. Numerical Solution of the TV flow
  6. Methods: TVflowNET
  7. Loss functional
  8. Joint space-time optimisation
  9. Architecture Design
  10. TVflowNET architectures
  11. Results
  12. Joint space-time optimisation
  13. TVflowNET
  14. Generalisability
  15. One-homogeneity
  16. ...and 3 more sections

Key Result

Theorem 1

Let $\mathcal{H}$ be a Hilbert space, $J:\mathcal{H} \to \mathbb{R} \cup \{+\infty\}$ a convex, lower semi-continuous and proper function, and $\partial J$ its subdifferential. For all $u_0\in\overline{\mathrm{dom}(\partial J)}$ there exists a unique function $u \in C([0,\infty);\mathcal{H})$ such t

Figures (8)

  • Figure 1: Example of the TV flow evolution. The TV flow solution at increasing time instances leads to more piecewise constant regions in the image. The initial image is taken from the STL-10 dataset Coates2011.
  • Figure 2: Visualisation of the TV flow solution $u$ derived using the TVflowNET with the U-Net architecture, the model-driven approach and the joint space-time optimisation. The example image is taken from the Food101 Food101 dataset.
  • Figure 3: Visualisation of $\text{div}\,\varphi$ derived using the TVflowNET with the U-Net architecture, the model-driven approach and the joint space-time optimisation. The example image is taken from the Food101 Food101 dataset.
  • Figure 4: TV flow solution of an image containing isolated disks derived using the TVflowNET with the U-Net architecture and the model-driven approach.
  • Figure 5: Evolution of the pixel value through the TV flow for two of the disks. Pixel values decrease linearly.
  • ...and 3 more figures

Theorems & Definitions (5)

  • Theorem 1: Brezis brezis1973ope
  • Theorem 2: Andreu-Vaillo2004parabolic
  • Corollary 3
  • proof
  • Theorem 4: Bredies2016