DeblurDiNAT: A Compact Model with Exceptional Generalization and Visual Fidelity on Unseen Domains

TL;DR

DeblurDiNAT tackles generalization to unseen real-world blur and perceptual quality in single-image deblurring by presenting a compact Transformer that combines alternating Dilated Neighborhood Attention (DiNA) with a channel-aware self-attention (CASA) framework. Key innovations include a local cross-channel learner (LCCL) integrated with DiNA, a lightweight linear divide-and-multiply FFN (DMFN), and a two-stage lightweight feature fusion (LDFF) for multi-scale information propagation. Empirical results show competitive or superior perceptual metrics on unseen domains while using only about 20% of the parameters of strong baselines, with qualitative gains in edge sharpness and texture fidelity. Ablation analyses confirm the contributions of hybrid dilation, CASA, DMFN, and LDFF, underscoring DeblurDiNAT’s effectiveness for robust, perceptually faithful deblurring in real-world scenarios.

Abstract

Recent deblurring networks have effectively restored clear images from the blurred ones. However, they often struggle with generalization to unknown domains. Moreover, these models typically focus on distortion metrics such as PSNR and SSIM, neglecting the critical aspect of metrics aligned with human perception. To address these limitations, we propose DeblurDiNAT, a deblurring Transformer based on Dilated Neighborhood Attention. First, DeblurDiNAT employs an alternating dilation factor paradigm to capture both local and global blurred patterns, enhancing generalization and perceptual clarity. Second, a local cross-channel learner aids the Transformer block to understand the short-range relationships between adjacent channels. Additionally, we present a linear feed-forward network with a simple while effective design. Finally, a dual-stage feature fusion module is introduced as an alternative to the existing approach, which efficiently process multi-scale visual information across network levels. Compared to state-of-the-art models, our compact DeblurDiNAT demonstrates superior generalization capabilities and achieves remarkable performance in perceptual metrics, while maintaining a favorable model size.

Figures (10)

• Figure 1: Deblurred results on the unseen RWBI dataset zhang2020deblurring, where mode collapse occurs in SOTA networks, UFPNet fang2023self and FFTformer kong2023efficient. In contrast, our proposed DeblurDiNAT resolves this issue while achieving comparable image quality scores with a minimal model size.
• Figure 2: Which deblurred image is clearer? The above example on RealBlur-J rim2020real suggests that ST-LPIPS ghildyal2022stlpips effectively captures visual fidelity in real-world scenarios, whereas the distortion metric PSNR fails, emphasizing the importance of using perceptual metrics.
• Figure 3: Architecture of DeblurDiNAT for single image deblurring. DeblurDiNAT is a hybrid encoder-decoder Transformer, where cascaded residual blocks extract multi-scale image features from input images, and the alternating local and global Transformer blocks reconstructs clean images from hierarchical features.
• Figure 4: Structures of (a) Transformer block, (b.1) local DiNA, (b.2) global DiNA, (c) local cross-channel learner (LCCL), and (d) divide-and-multiply feed-forward network (DMFN) in DeblurDiNAT. Note: GAP represents global average pooling; PConv and Dconv refer to point-wise and depth-wise convolutions; Conv1D denotes the convolution applied to 1D sequences. Tensor transformations are omitted for simplicity.
• Figure 5: Structures of (b) lightweight dual-stage feature fusion (LDFF), composed of (b.1) efficient channel reduction (ECR) and (b.2) complementary feature mixer (CFM).