Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decomposition Ascribed Synergistic Learning for Unified Image Restoration

Jinghao Zhang, Feng Zhao

TL;DR

The diverse degradations are revisited through the lens of singular value decomposition, with the observation that the decomposed singular vectors and singular values naturally undertake the different types of degradation information, dividing various restoration tasks into two groups, \ie, singular vector dominated and singular value dominated.

Abstract

Learning to restore multiple image degradations within a single model is quite beneficial for real-world applications. Nevertheless, existing works typically concentrate on regarding each degradation independently, while their relationship has been less exploited to ensure the synergistic learning. To this end, we revisit the diverse degradations through the lens of singular value decomposition, with the observation that the decomposed singular vectors and singular values naturally undertake the different types of degradation information, dividing various restoration tasks into two groups, \ie, singular vector dominated and singular value dominated. The above analysis renders a more unified perspective to ascribe the diverse degradations, compared to previous task-level independent learning. The dedicated optimization of degraded singular vectors and singular values inherently utilizes the potential relationship among diverse restoration tasks, attributing to the Decomposition Ascribed Synergistic Learning (DASL). Specifically, DASL comprises two effective operators, namely, Singular VEctor Operator (SVEO) and Singular VAlue Operator (SVAO), to favor the decomposed optimization, which can be lightly integrated into existing image restoration backbone. Moreover, the congruous decomposition loss has been devised for auxiliary. Extensive experiments on blended five image restoration tasks demonstrate the effectiveness of our method.

Decomposition Ascribed Synergistic Learning for Unified Image Restoration

TL;DR

The diverse degradations are revisited through the lens of singular value decomposition, with the observation that the decomposed singular vectors and singular values naturally undertake the different types of degradation information, dividing various restoration tasks into two groups, \ie, singular vector dominated and singular value dominated.

Abstract

Learning to restore multiple image degradations within a single model is quite beneficial for real-world applications. Nevertheless, existing works typically concentrate on regarding each degradation independently, while their relationship has been less exploited to ensure the synergistic learning. To this end, we revisit the diverse degradations through the lens of singular value decomposition, with the observation that the decomposed singular vectors and singular values naturally undertake the different types of degradation information, dividing various restoration tasks into two groups, \ie, singular vector dominated and singular value dominated. The above analysis renders a more unified perspective to ascribe the diverse degradations, compared to previous task-level independent learning. The dedicated optimization of degraded singular vectors and singular values inherently utilizes the potential relationship among diverse restoration tasks, attributing to the Decomposition Ascribed Synergistic Learning (DASL). Specifically, DASL comprises two effective operators, namely, Singular VEctor Operator (SVEO) and Singular VAlue Operator (SVAO), to favor the decomposed optimization, which can be lightly integrated into existing image restoration backbone. Moreover, the congruous decomposition loss has been devised for auxiliary. Extensive experiments on blended five image restoration tasks demonstrate the effectiveness of our method.
Paper Structure (22 sections, 2 theorems, 10 equations, 92 figures, 11 tables)

This paper contains 22 sections, 2 theorems, 10 equations, 92 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Method
  4. Overview
  5. Singular Vector Operator
  6. Singular Value Operator
  7. Optimization objective
  8. Experiments
  9. Implementation Details
  10. Comparison with state-of-the-art methods
  11. Abalation Studies
  12. Limitation and Future works
  13. Conclusion
  14. Proof of the Propositions
  15. Proof of Theorem 1
  16. ...and 7 more sections

Key Result

Theorem 3.1

For an arbitrary matrix $X \in \mathbb{R}^{h\times w}$ and random orthogonal matrices $P \in \mathbb{R}^{h\times h}, Q\in \mathbb{R}^{w\times w}$, the products of the $PX$, $XQ$, $PXQ$ have the same singular values with the matrix $X$.

Figures (92)

  • Figure 1: An illustration of the decomposition ascribed analysis on various image restoration tasks through the lens of the singular value decomposition. The decomposed singular vectors and singular values undertake the different types of degradation information as we recompose the degraded image with portions of the clean counterpart, ascribing diverse restoration tasks into two groups, i.e., singular vector dominated rain, noise, blur, and singular value dominated low-light, haze. Dedicated to the decomposed optimization of the degraded singular vectors and singular values rendering a more unified perspective for synergistic learning, compared to previous task-level independent learning.
  • Figure 2: The statistic validation that the decomposed singular vectors and singular values undertake the different types of degradation information. (a) The reconstruction error between the recomposed image and paired clean image on five common image restoration tasks. Low error denotes the degradation primarily distributed in the replaced portion of the image. (b) The boxplot comparison of singular value distribution between the degraded image and corresponding clean image, where the singular value dominated low-light and haze exhibit extraordinary difference. (c) The singular vector difference on separate orders of the component between the degraded image and clean image, where the singular vector dominated rain, noise, and blur present more disparity. The results are obtained under calculation on 100 images for each restoration task.
  • Figure 3: An illustration of the proposed Singular Vector Operator (SVEO), which is dedicated on the optimization of the singular vector dominated degradations, i.e. rain, noise, blur. Theorem \ref{['theorem:orth']} supports the feasibility and the orthogonal regularization $\mathcal{L}_{orth}$ refers to Eq. \ref{['eq:orth']}.
  • Figure 4: An illustration of the core idea of the proposed Singular Value Operator (SVAO), which is dedicated on the optimization of the singular value dominated degradations, i.e. haze and low-light. Two-dimensional signal formations are provided for simplicity.
  • Figure 5: Visual comparison of the progressive reconstruction results with SVD and IDFT components, respectively. First row, IDFT reconstruction result. Second row, SVD reconstruction result. Both conform to the principle from outline to details.
  • ...and 87 more figures

Theorems & Definitions (2)

  • Theorem 3.1
  • Theorem 1.1