Correlation Matching Transformation Transformers for UHD Image Restoration
Cong Wang, Jinshan Pan, Wei Wang, Gang Fu, Siyuan Liang, Mengzhu Wang, Xiao-Ming Wu, Jun Liu
TL;DR
UHDformer tackles UHD image restoration under challenging conditions by enabling learning in both high- and low-resolution spaces and introducing a high-to-low feature transformation. The core innovations are DualCMT, which selects top $C/r$ correlation channels from high-resolution features to refine low-resolution representations, and ACM, which adaptively modulates multi-level high-resolution features to supply richer content to the low-resolution branch. Together, these components yield a general Transformer framework that substantially reduces model sizes (up to about 98% fewer parameters) while delivering consistent PSNR/SSIM improvements across three UHD restoration tasks: low-light image enhancement, image dehazing, and image deblurring. The approach enables efficient UHD restoration suitable for devices with limited resources and provides a robust, task-agnostic transformer design for UHD content with a strong practical impact.
Abstract
This paper proposes UHDformer, a general Transformer for Ultra-High-Definition (UHD) image restoration. UHDformer contains two learning spaces: (a) learning in high-resolution space and (b) learning in low-resolution space. The former learns multi-level high-resolution features and fuses low-high features and reconstructs the residual images, while the latter explores more representative features learning from the high-resolution ones to facilitate better restoration. To better improve feature representation in low-resolution space, we propose to build feature transformation from the high-resolution space to the low-resolution one. To that end, we propose two new modules: Dual-path Correlation Matching Transformation module (DualCMT) and Adaptive Channel Modulator (ACM). The DualCMT selects top C/r (r is greater or equal to 1 which controls the squeezing level) correlation channels from the max-pooling/mean-pooling high-resolution features to replace low-resolution ones in Transformers, which can effectively squeeze useless content to improve the feature representation in low-resolution space to facilitate better recovery. The ACM is exploited to adaptively modulate multi-level high-resolution features, enabling to provide more useful features to low-resolution space for better learning. Experimental results show that our UHDformer reduces about ninety-seven percent model sizes compared with most state-of-the-art methods while significantly improving performance under different training sets on 3 UHD image restoration tasks, including low-light image enhancement, image dehazing, and image deblurring. The source codes will be made available at https://github.com/supersupercong/UHDformer.