Unified-Width Adaptive Dynamic Network for All-In-One Image Restoration

TL;DR

The paper tackles all-in-one image restoration under diverse and unknown degradations by introducing the Unified-Width Adaptive Dynamic Network (U-WADN). U-WADN combines a Width Adaptive Backbone (WAB), which contains nested sub-networks of increasing width, with a Width Selector (WS) that dynamically assigns input samples to an appropriate width based on task- and sample-wise complexity. Through a reformulation that views complex tasks as simpler tasks plus residual degradations, and via staged training with reconstruction, distillation, and degradation-encoding losses, U-WADN achieves higher PSNR/SSIM while reducing FLOPs by up to 32.3% and speeding up processing by about 15.7% across five restoration tasks. Ablation studies show the benefits of the WS and the WAB components, and experiments demonstrate competitive performance with substantially improved efficiency, highlighting U-WADN’s potential for practical, real-time all-in-one restoration deployments.

Abstract

In contrast to traditional image restoration methods, all-in-one image restoration techniques are gaining increased attention for their ability to restore images affected by diverse and unknown corruption types and levels. However, contemporary all-in-one image restoration methods omit task-wise difficulties and employ the same networks to reconstruct images afflicted by diverse degradations. This practice leads to an underestimation of the task correlations and suboptimal allocation of computational resources. To elucidate task-wise complexities, we introduce a novel concept positing that intricate image degradation can be represented in terms of elementary degradation. Building upon this foundation, we propose an innovative approach, termed the Unified-Width Adaptive Dynamic Network (U-WADN), consisting of two pivotal components: a Width Adaptive Backbone (WAB) and a Width Selector (WS). The WAB incorporates several nested sub-networks with varying widths, which facilitates the selection of the most apt computations tailored to each task, thereby striking a balance between accuracy and computational efficiency during runtime. For different inputs, the WS automatically selects the most appropriate sub-network width, taking into account both task-specific and sample-specific complexities. Extensive experiments across a variety of image restoration tasks demonstrate that the proposed U-WADN achieves better performance while simultaneously reducing up to 32.3\% of FLOPs and providing approximately 15.7\% real-time acceleration. The code has been made available at \url{https://github.com/xuyimin0926/U-WADN}.

Figures (8)

• Figure 1: Overall pipeline of the proposed Unified-Width Adaptive Dynamic Network. (a) Unified-Width Adaptive Dynamic Network (U-WADN); (b) Width-Adaptive Convolution.
• Figure 2: The Width Selector (WS) in the proposed U-WADN.
• Figure 3: Visual comparison among methods on image denoising. (a) is the input noisy image, and from (b) to (g) are results from CBM3D dabov2007color, DnCNN zhang2017beyond, IRCNN zhang2017learning, FFDNet zhang2018ffdnet, AirNet li2022all and U-WADN, respectively. (h) is the ground truth of clean images.
• Figure 4: Visual comparison among methods on image deraining. (a) is the input rainy image, and from (b) to (g) are results from DIDMDN zhang2018density, UMRL yasarla2019uncertainty, SIRR wei2019semi, LPNet fu2019lightweight, AirNet li2022all, and U-WADN, respectively. (h) is the ground truth of clean images.
• Figure 5: Visual comparison among methods on image dehazing. (a) is the input rainy image, and from (b) to (g) are results from DehazeNet cai2016dehazenet, AOD-Net li2017aod, EPDN qu2019enhanced, FDGAN dong2020fd, AirNet li2022all, and U-WADN, respectively. (h) is the ground truth of clean images.