Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration

Jiahong Fu, Qi Xie, Deyu Meng, Zongben Xu

TL;DR

A high-accuracy rotation equivariant proximal network that effectively embeds rotation symmetry priors into the deep unfolding framework is suggested and the theoretical equivariant error for such a designed proximal network with arbitrary layers under arbitrary rotation degrees is deduced.

Abstract

The deep unfolding approach has attracted significant attention in computer vision tasks, which well connects conventional image processing modeling manners with more recent deep learning techniques. Specifically, by establishing a direct correspondence between algorithm operators at each implementation step and network modules within each layer, one can rationally construct an almost ``white box'' network architecture with high interpretability. In this architecture, only the predefined component of the proximal operator, known as a proximal network, needs manual configuration, enabling the network to automatically extract intrinsic image priors in a data-driven manner. In current deep unfolding methods, such a proximal network is generally designed as a CNN architecture, whose necessity has been proven by a recent theory. That is, CNN structure substantially delivers the translational invariant image prior, which is the most universally possessed structural prior across various types of images. However, standard CNN-based proximal networks have essential limitations in capturing the rotation symmetry prior, another universal structural prior underlying general images. This leaves a large room for further performance improvement in deep unfolding approaches. To address this issue, this study makes efforts to suggest a high-accuracy rotation equivariant proximal network that effectively embeds rotation symmetry priors into the deep unfolding framework. Especially, we deduce, for the first time, the theoretical equivariant error for such a designed proximal network with arbitrary layers under arbitrary rotation degrees. This analysis should be the most refined theoretical conclusion for such error evaluation to date and is also indispensable for supporting the rationale behind such networks with intrinsic interpretability requirements.

Rotation Equivariant Proximal Operator for Deep Unfolding Methods in Image Restoration

TL;DR

A high-accuracy rotation equivariant proximal network that effectively embeds rotation symmetry priors into the deep unfolding framework is suggested and the theoretical equivariant error for such a designed proximal network with arbitrary layers under arbitrary rotation degrees is deduced.

Abstract

The deep unfolding approach has attracted significant attention in computer vision tasks, which well connects conventional image processing modeling manners with more recent deep learning techniques. Specifically, by establishing a direct correspondence between algorithm operators at each implementation step and network modules within each layer, one can rationally construct an almost ``white box'' network architecture with high interpretability. In this architecture, only the predefined component of the proximal operator, known as a proximal network, needs manual configuration, enabling the network to automatically extract intrinsic image priors in a data-driven manner. In current deep unfolding methods, such a proximal network is generally designed as a CNN architecture, whose necessity has been proven by a recent theory. That is, CNN structure substantially delivers the translational invariant image prior, which is the most universally possessed structural prior across various types of images. However, standard CNN-based proximal networks have essential limitations in capturing the rotation symmetry prior, another universal structural prior underlying general images. This leaves a large room for further performance improvement in deep unfolding approaches. To address this issue, this study makes efforts to suggest a high-accuracy rotation equivariant proximal network that effectively embeds rotation symmetry priors into the deep unfolding framework. Especially, we deduce, for the first time, the theoretical equivariant error for such a designed proximal network with arbitrary layers under arbitrary rotation degrees. This analysis should be the most refined theoretical conclusion for such error evaluation to date and is also indispensable for supporting the rationale behind such networks with intrinsic interpretability requirements.
Paper Structure (16 sections, 3 theorems, 22 equations, 11 figures, 7 tables)

This paper contains 16 sections, 3 theorems, 22 equations, 11 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Deep Unfolding Network for Image Restoration
  4. Group Equivariant CNNs
  5. Equivariant Proximal Network
  6. General Framework of Deep Unfolding
  7. A Principle for Proximal Network Design
  8. Rotation Equivariant Proximal Network
  9. Selection of Equivariant Convolutions
  10. Theoretical Analysis for Equivariant CNN
  11. Experimental results
  12. Blind Single Image Super-Resolution
  13. Metal Artifact Reduction in CT Image
  14. Single Image Rain Removal
  15. Comparison of Number of Parameters
  16. ...and 1 more sections

Key Result

Lemma 1

Suppose that $\mathcal{X}$ is a Hilbert space and $\pi_g: \mathcal{X} \rightarrow \mathcal{X}$ is a unitary transformation of a group $G$ on $\mathcal{X}$. If a functional $R: \mathcal{X} \rightarrow \mathbb{R}\cup \{+\infty\}$ has a well-defined single-valued proximal operator $\operatorname{prox}_

Figures (11)

  • Figure 1: Illustration of the output feature map of a typical image obtained by standard CNN and our used rotation equivariant convolution neural network (E-CNN).
  • Figure 2: The values of four typical conventional regularization terms on the Lena image with different rotation angles.
  • Figure 3: Illustrations of the feature maps and outputs of different CNNs when rotating the input images for $2k\pi/t$, where $k = 1,2,\cdots, t$, $t$ is the equivariant number. (a) A standard CNN. (b) The rotation equivariant CNN with the same backbone.
  • Figure 4: Performance comparison img 095 in Urban100 huang2015single, where the Gaussian noise is with standard deviation of 50.
  • Figure 5: Performance comparison on img 004 in Urban100 huang2015single. The scale factor is 4 and the standard deviation is 5.
  • ...and 6 more figures

Theorems & Definitions (3)

  • Lemma 1
  • Lemma 2
  • Theorem 1