Deep RAW Image Super-Resolution. A NTIRE 2024 Challenge Survey
Marcos V. Conde, Florin-Alexandru Vasluianu, Radu Timofte, Jianxing Zhang, Jia Li, Fan Wang, Xiaopeng Li, Zikun Liu, Hyunhee Park, Sejun Song, Changho Kim, Zhijuan Huang, Hongyuan Yu, Cheng Wan, Wending Xiang, Jiamin Lin, Hang Zhong, Qiaosong Zhang, Yue Sun, Xuanwu Yin, Kunlong Zuo, Senyan Xu, Siyuan Jiang, Zhijing Sun, Jiaying Zhu, Liangyan Li, Ke Chen, Yunzhe Li, Yimo Ning, Guanhua Zhao, Jun Chen, Jinyang Yu, Kele Xu, Qisheng Xu, Yong Dou
TL;DR
This work surveys the NTIRE 2024 RAWSR Challenge, tackling RAW Bayer image upscaling by $2\times$ under unknown degradations to advance ISP-aware restoration. It highlights a BSRAW-based degradation framework, a 1064-image RAW training set, and three testing splits (validation, 1MP, and full 12MP RAW outputs), with PSNR/SSIM as fidelity metrics. The paper reviews five top methods—dual-stage networks with focal pixel loss, effective RAWSR with dual branches, transformer-based RAWSR variants, SwinIR-inspired architectures, and SAFM-FFT with knowledge distillation—each employing distinct degradation models and training strategies. Key findings indicate substantial RAW-domain gains over baselines and reveal that synthetic RAW degradation pipelines can close much of the gap in RAWSR, while real-world downsampling still poses a challenge. The results underscore the potential for RAW SR to enhance ISP pipelines on consumer and professional devices, guiding future efforts toward more realistic degradations and efficient architectures.
Abstract
This paper reviews the NTIRE 2024 RAW Image Super-Resolution Challenge, highlighting the proposed solutions and results. New methods for RAW Super-Resolution could be essential in modern Image Signal Processing (ISP) pipelines, however, this problem is not as explored as in the RGB domain. Th goal of this challenge is to upscale RAW Bayer images by 2x, considering unknown degradations such as noise and blur. In the challenge, a total of 230 participants registered, and 45 submitted results during thee challenge period. The performance of the top-5 submissions is reviewed and provided here as a gauge for the current state-of-the-art in RAW Image Super-Resolution.