Xformer: Hybrid X-Shaped Transformer for Image Denoising

TL;DR

Xformer tackles image denoising by enhancing global token representations through a dual-branch Transformer that processes spatial-wise and channel-wise interactions in parallel. Each branch uses encoder–decoder Transformer blocks with a Bidirectional Connection Unit to fuse cross-branch information, yielding robust multi-scale representations and the final restoration $\hat{I} = I + I_r$ where $I_r$ is the learned residual. The approach delivers state-of-the-art PSNR/SSIM on Gaussian and real-world denoising with competitive complexity compared to Restormer and SwinIR, and code is released for reproducibility. This work demonstrates the effectiveness of jointly modeling patch-level and channel-level information in a concurrent X-shaped architecture for low-level vision tasks.

Abstract

In this paper, we present a hybrid X-shaped vision Transformer, named Xformer, which performs notably on image denoising tasks. We explore strengthening the global representation of tokens from different scopes. In detail, we adopt two types of Transformer blocks. The spatial-wise Transformer block performs fine-grained local patches interactions across tokens defined by spatial dimension. The channel-wise Transformer block performs direct global context interactions across tokens defined by channel dimension. Based on the concurrent network structure, we design two branches to conduct these two interaction fashions. Within each branch, we employ an encoder-decoder architecture to capture multi-scale features. Besides, we propose the Bidirectional Connection Unit (BCU) to couple the learned representations from these two branches while providing enhanced information fusion. The joint designs make our Xformer powerful to conduct global information modeling in both spatial and channel dimensions. Extensive experiments show that Xformer, under the comparable model complexity, achieves state-of-the-art performance on the synthetic and real-world image denoising tasks. We also provide code and models at https://github.com/gladzhang/Xformer.

Figures (5)

• Figure 1: Architecture of our proposed Xformer. The modules include spatial-wise Transformer block (STB), channel-wise Transformer block (CTB), and bidirectional connection unit (BCU).
• Figure 2: Visual examples for Gaussian color image denoising with noise level $\sigma$=50 on Urban100.
• Figure 3: Module architectures of spatial-wise and channel-wise Transformer blocks (STB|CTB).
• Figure 4: Visualization of feature maps from the last STB and CTB in the encoder-decoder module of dual branches. We compare different situations about whether to use BCU.
• Figure 5: Visual comparisons on Gaussian color and gray image denoising ($\sigma$=50).